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ABSTRACT 

This guide suggests processes for making informed 
software selection, describes several prominent software evaluation 
models, raises key issues in the software selection process, and 
provides information on a variety of print and organizational 
resources. The first section, featuring an article by Henry F. Olds, 
Jr., "Evaluating the Evaluation Schemes ," presents several key 
principles for educators to consider in software evaluation. The 
second section contains brief summaries of ten software evaluation 
models from organizations representing both non-profit, cooperative 
systems, and private companies: (1) MicrooIFT (Microcomputer Software 
and Information for Teachers); (2) EPIE (Educational Products 
Information Exchange) and Consumers Union; (3) "School Microware 
Reviews"; (4) "Courseware Report Card"; (5) Minnesota Educational 
Computing Consortium (MECC) ; (6) SOFTSWAP (San Mateo Educational 
Resources Center); (7) CONDUIT; (8) National Council of Teachers of 
Mathematics; (9) Scholastic Book Services? and (10) "Electronic 
Learning." Several of the summaries describe overall strategies for 
software evaluation, procedures, criteria, and rating systems. The 
third section includes sample software reviews that show the results 
of applying criteria and ratings on various dimensions of educational 
software. The final section provides several resource lists focusing 
on various aspects of software availability and evaluation. The 
listings include books, journal articles, periodicals, directories, 
and other resources . ( LMM) 
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Foreword 

Computer-instruction has become a regular feature in classrooms across the country. The successful use 
of computers to augment classroom instruction requires that teachers learn several new skills In 
particular, educators are seeking help in the identification of software that can best serve their needs 
So^wovf 8011 ' Regional Exchange at the Southwest Educational Development Laboratory, 
(SEDL/RX) and the Northeast Regional Exchange, Inc. (NEREX) have joined forces to produce a 
resource book abou t software evaluation. By collab orating in the de /eloprnent of this book, not only have 
both organizations benefited, but combining the resources of two regions has greatly expanded the 
scope of available information aboutsoftware evaluation. The editors, Nancy Baker Jones of the SEDL 
Regional Exchange and Larry Vaughan of the NEREX staff, deserve praise for the result of their 
collaboration. In their work, the editors successfully engaged the talents and perspectives of several key 
education professionals in both regions, as well as from the national Research and Development 
Exchange network' (RDx). 

The Northeast Regional Exchange, and the Regional Exchange of the Southwest Educational Develop- 
ment Laboratory are indebted to many people for their assistance in the development of this book We 
are particularly grateful for the assistance of the members of the NEREX Instructional Technology 
Task Force and the SEDL/RX Advisory Board. The Task Force helped to mold the overall design for 
this book and contributed much of the resource information in the appendices, while the SEDL/RX 
Advisory Board provided the impetus for SEDL's involvement ir. producing tnis book and provided 
support throughout the process. Members of the NEREX Task Force are: 

Elizabeth Glass, Connecticut; Richard Riley, Maine; Roselyn Fiank, Paul Maloney, Clark Adams 
and Susan Foote, Massachusetts; Femand Prevost, New Hampshire; Norman Kurland Robert 
T ! rombly, and GregBenson, New York; Harry Darling, andRobertRude, Rhode Island; andRobert 
Kenney, Vermont. 

Members of the SEDL/RX Advisory Board are: 

Sara Murphy, Arkansas; Sue Wilson, Louisiana; Clyde Hatten, Mississippi; Susan Brown New 
Mexico; Jack Craddock, Oklahoma; Marj Wri^htman, and J.S. Miguel, Texas. 

Henry F. Olds, Jr. author of "Evaluating the Evaluation Schemes," contributed to the development of 
this book by providing information on resource articles, linkage with key people, and insights about the 
key issues involved in software evaluation. 

Many educators in our respective regions have generously given their time to share their concerns for 
software evaluation. We are grateful to them and to all who so generously gave their time and materials 
for inclusion in this publication. 

The editorial board for Evaluation of Educational Software: A Guide to Guides is also due applause. Its 
members - Vicki Blum Cohen, Director of Instructional Design for ISO Communications, Inc., and 
formerly with EPIE and Columbia University's Microcomputer Resource Center; Donald Holznagel 
CoordinatorofMicroSIFT.NorthwestRegionalEducationalLaboratory; and JamesPoirot, Chairof the 
Department of Computer Sciences, North Texas State University - assisted the editors by recom- 
mending the best resources from thu rather large world of information on software evaluation. They also 
reviewed and commented on the final draft of the book. 

For production of this book, we owe thanks to Gloria Bowles of the NEREX staff, who was involved in 
development and refinement of this book from the beginning. Dennis Collins of LEA Associates in 
Concord, New Hampshire, was instrumental in preparing the document for printing, and Desiree Burke, 
an Austin artist, designed the cover. 

Preston C. Kronkosky J. Lynn Griesemer 

Executive Director Executive Director 

SEDL NEREX 
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Current surveys from virtually every section of the 
country reveal a variety of efforts to augment class- 
room instruction through the use of computers. 
Although a few schools have been using computer 
instruction for more than a decade, many schools 
have just begun, now that the microcomputer has 
made computer instruction more feasible. The 
number of microcomputers in schools is growingat 
a rate approaching geometric progression. Educa- 
tors face a variety of problems as they attempt to 
integrate computer instruction in schools. In par- 
ticular, there are important decisions on the alloca- 
tion of limited resources for computer instruction. 
The selection of software to support computer 
instruction often represents a considerable invest- 
ment, and these investment decisions continue to 
be complicated by the dynamics of a rapidly de- 
veloping field. Good information is needed by edu- 
cators to help guide the software selection process. 
More and more information about software is be- 
coming available from a variety of different sour- 
ces, but it is difficult to locate and keep track of 
reliable sources of information about the selection 
of software. 

Our overall goal in developing Evaluation of Edu- 
cational Software: A Guide to Guides is to provide 



awareness, understanding, and reference infor- 
mation about the evaluation of software. Speci- 
fically, this document is intended to: 

• present suggested processes for making in- 
formed software selections 

• provide awareness of several prominent models 
for software evaluation 

• raise key issues in the software selection \ o- 
cess, and 

• provide information on a variety of print and 
organizational resources that may be useful to 
those making software decisions. 

This book is designed to serve several educational 
audiences. Individual administrators and teachers 
who have responsibility for software selection, 
members of school-based courseware selection 
committees, pre-service and inservice students 
who are forming opinions aboutsoftware, and edu- 
cators who are developing their own software can 
use this publication in a variety of ways. Informa- 
tion in the following pages may help guide the 
development or refinement of a local philosophy 
and policies regarding software evaluation. Des- 
criptions of software evaluation models and sys- 
tems may be used to help determine what sources 
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of information should be regularly accessed to 
inform local decision making. The procedures, cri- 
teria, and forms from these evaluation models can 
be adapted for local applications. Also, the refer- 
ence materials and organizations cited can guide 
educators as they seek specific types of informa- 
tion about software availability and evaluation. We 
hope that commercial software developers will 
make use of the information contained in this book 
ns they develop new instructional software. 

This book has four main sections. The first sec- 
tion, featuring an article by Henry F. Olds, Jr., 
"Evaluating the Evaluation Schemes," presents 
several key principles for educators to consider in 
the evaluation of software. The final responsibility 
for selecting computer software rests with local 
decision makers. These educators have the respon- 
sibility for knowing what constitutes appropriate 
software for various instructional computing needs. 
The major organizations engaged in software eval- 
uation described in this book provide useful ser- 
vices. These organizations have identified a variety 
of issue categories pertinent to software evalua- 
tion, developed evaluation criteria, developed re- 
view and evaluation forms that can be adapted for 
local uses, and provided descriptive and evalua- 
tive comments about specific software packages. 
We recommend that educators not rely too heavily 
on software evaluation conducted by persons out- 
side of the local school system. No evaluation ser- 
vice or form, by itself, can serve the real purpose of 
evaluating software: to find pedagogically sound 



curricula materials appropriate for the learner and 
the lesson. Published reviews and evaluations help 
to identify software that can be more carefully 
examined at the local level in light of local cri- 
teria. 

The second section of this publication contains 
brief summaries of ten major models for software 
evaluation. The ten organizations included repre- 
sent both non-profit, cooperative systems, and 
private companies. Several of the summaries de- 
scribe overall strategies for software evaluation, 
procedures, criteria, and rating systems. We re- 
commend that interested educators find out more 
about these organizations and adapt those pro- 
cedures, criteria, and strategies that appear most 
in line with their own software evaluation prior- 
ities. 

The next section includes sample software re- 
views. These show the results of applying criteria 
and ratings on various dimensions of educational 
software. The final section of this book contains 
several resource lists focusing on various aspects 
of software availability and evaluation. New, per- 
tinent print and organizational resources are be- 
coming available as the field of computer instruc- 
tion expands and develops. The books, journal 
articles, periodicals, directories, snd other resour- 
ces listed here were valuable to the editors in de- 
veloping this publication. We recommend that edu- 
cators investigate these and other resources as 
they develop the local capacity to review, evaluate, 
and select educational software. 
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Toward a New Perspective on 
Software Evaluation 

The value of any computer lies in the value of the 
software available for it. While the microprocessor 
has made computers affordable for use in homes 
and classrooms, purchasing a microcomputer or, 
more often, several microcomputers, is still a sub- 
stantial investment. That investment is relatively 
worthless without the software that makes the 
microcomputeruseful. Indeed, the software invest- 
ment required to use the computer may exceed the 
original investment in the computer itself. 

Now that there is increasingly more sophistication 
about how this technology can be appropriately 
applied to education, we know that a very large 
percentage (some say as high as 90%) of educa- 
tional software currently on the market is not 
worth buying. Much of it consists of misguided 
efforts to translate textbook materials into soft- 
ware. In other cases, the software is poorly con- 
ceived, poorly designed, and difficult to use. Soft- 
ware that combines the best efforts of experienced 
educators and expert programmers is only just 
beginning to reach the market in any quantity. 

There are a few general features that characterize 
quality software: 

1) It should be the outgrowth of a fully con- 
ceived and carefully articulated "intellectual 
model of the content domain" (e.g., good science 
software should represent the best current scien- 
tific understanding). 



2) It should reflect an understanding of the cog- 
nitive developmental needs and capacities of the 
learner (e.g., younger children need more concrete 
representational experiences). 

3) Since the computer provides an environ- 
ment for interaction, its best pedagogical use should 
support the inherently interactive nature of know- 
ledge construction for the learner (e.g., the com- 
puter is a poor medium for giving a lecture). 

4) It should make use of the special qualities of 
computertechnology in truly functional ways (e.g., 
use graphics to make an abstract concept concrete). 
It should not attempt to carry out instructional 
tasks far better suited to other media (e.g., it 
should not be a textbook). 

Each of these features can be much further refined. 
While the first two also apply to non-computer 
instructional materials, they must be reconsidered 
in the lightof this new technology. The last two are 
the areas where most current attention is now 
focused. 

The size and importance of the investment in com- 
puters and the lackof quality software make choos- 
ing good software a critical issue for e iucators 
today. This issue is complicated by the lack of a 
shared language to communicate about a new tech- 
nology that has the capacity to provide a highly 
multi-sensory experience in a thoroughly interac- 
tive fashion. Few educators yet are able to talk 
knowledgeably about this new mode of instruction 
with which they have had almost no direct or ex- 
tensive experience. Unfortunately, the experiences 
of most have been indirect (hearing or reading 
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aboutcompu ting, not doing it) and limited to only a 
few moments with only one kind of software. 

Nevertheless, the question we must concern our- 
selves with is: how can people be helped to make 
good choices in a new domain where there is little 
shared experience and no shared language? One 
response to this question is to create software 
evaluation forms that can be filled out by experi- 
enced computer-using educators. The evaluations 
can then be synthesized and reported to others 
seeking useful information in a way that contri- 
butes to making good software choices. 

On the surface, this approach sounds reasonable 
and worthwhile. Very serious, well-intentioned ef- 
forts have been made to develop evaluation for- 
mats that are both informative and useful, and 
these efforts merit our attention. A number of 
them are presented in this book. They are infor- 
mative both because they lay out much of the 
domain that must be considered and because they 
illustrate many of the dilemmas inherent in this 
form of evaluation. 

Any form of evaluation might be expected to have 
the following features: 

1) Since the primary function of a good evalua- 
tion is to inform, software should be described in 
detail before it is evaluated. 

2) Evaluation should be qualitative, not quan- 
titative. Good and bad features should be discus- 
sed, and rating scales should be kept to a min- 
imum. Numbers should not be relied upon to con- 
vey important substantive matters. 

3) Evaluations should be timely; they should 
treat recently published products shortly after ini- 
tial publication. 

4) The evaluation should compare the program 
under consideration with others of a similar type. 

5) Evaluations should distinguish between those 
aspects of design that may be easily remedied and 
those which cannot be changed without overall 
reconceptualization. 

Description Before Evaluation 

In Minds torms, Seymour Papert says, "The com- 
puter is the Proteus of machines. Its essence is its 
universality, its power to simulate. Because it can 
take on a thousand forms and serve a thousand 
functions, it can appeal to a thousand tastes. 

The thousands of forms that a computer can be- 
come are the products of the software created for 
it. To understand the computer, therefore, is to be 
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able to describe its software. The most important 
task that must be undertaken, before attempting 
the evaluation of any particular program, is to 
create a general descriptive framework for soft- 
ware that can be used to make as clear as possible 
what a software program is and what it is not. 

Just as it does not make any sense to evaluate a 
meal of beef bourguignonne at a fine French res- 
taurant in the same terms as a hamburger at a fast 
food outlet, even though they can both be called 
food, it does not make sense to evaluate a drill and 
practice program on math facts in the same terms 
as a database program, even though they are both 
software. It is terribly important that people begin 
to understand that software is probably no less 
diverse than food. Furthermore, there is "junk 
software'' — software of dubious m erit that should, 
perhaps, have a warning attached to it about pos- 
sible "health hazards," and there is "gourmet 
software" — perhaps beyond the average person's 
everyday tastes or needs but which, nonetheless, 
sets high standards of quality for all other soft- 
ware. 

A commonly accepted way of describing software 
(a descriptive taxonomy) is badly needed. It must 
not be a rigid framework because software devel- 
opment is still in its infancy, and any framework 
must be adaptable to new (and undoubtedly sur- 
prising) creations that don't quite fit. Severalyears 
ago, the Cognitive Research Group at Education 
Development Center (Newton, MA) developed a 
beginning framework as part of an intensive study 
of computer software supported by Control Data 
Corporation (Olds, Schwartz and Willie, "People 
and Computers: Who Teaches Whom?"). As a reg- 
ular software reviewer for two publications (Class- 
room Computer News and WINDOW), I have found 
this framework extremely helpful in understand- 
ing how to think about software and in describing it 
to my readers. As a trainer of teachers, I have 
found this framework of tremendous value for de- 
veloping a shared language to talk about software. 

Over a couple ofyears, I have modified the original 
framework slightly and changed the language a bit 
here and there. The framework places software 
somewhere on a continuum: from plain and un- 
sophisticated applications (fast food) to fancy and 
highly sophisticated applications (gourmet). It 
must be emphasized that this is not an evaluative 
framework — just as there can be very good and 
healthy fast food, which can be just the right food 
on certain occasions, so there can be very good and 
effective software that is very plain and unsophis- 
ticated. 
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A Descriptive Framework 

It is helpful to distinguish between three broad 
categories of software, though in reality the dis- 
tinctions sometimes get a bit fuzzy at the boun- 
daries. First, there is software that uses the com- 
puter as a medium to transmit information or to 
instruct (often referred to as CAI — computer 
assisted instruction). Then there is software that 
uses the computer as a modeling device for creating 
an environment with which the user interacts. And 
finally, there is software that uses the computer as 
a tc>l with which the user performs some task. 
Each of these major categories has some impor- 
tant sub-categories, which we shall now describe 
briefly. 

/. Computer as Instructional Medium 
(CAI) 

A. Drill and Practice — the use of the computer 
to evoke continued and improved performance in 
some well-specified skill or knowledge domain. 
The computer can provide potentially limitless 
practice exercises and immediate feedback to the 
user. Effective applications of drill and practice 
software have been made in math number facts, 
spelling, foreign language vocabulary, typing, etc. 

B. Tutorials — the use of the computer to teach 
some subject matter directly. At their best, some 
tutorials use the computer to respond to the user 
by adjusting both the feedback and the continuing 
instruction to the user's growing understanding (a 
process known as branching) . A fast learner moves 
quickly through the material, while a slower learner 
is given several alternative opportunities to learn. 
As with drill and practice, tutorials tend to be most 
effective when the content being taught can be 
very clearly specified. They also work best when a 
high level of motivation canbe assumed on the part 
of the user. 

D. Computer as Modeling Device 

A. Games — the use of the computer to model 
an interactive environment in which the user is 
required tooutmaneuver, outthink,or outwitother 
users or the computer. Games challenge the user 
to reach a full enough understanding of theirstruc- 
ture to master playing them. Thus, they may en- 
gage and help to develop a wide range of problem- 
solving skills. Though game elements are fre quently 
used to enliven drill and practice ortutorials, pure 
games rarely teach directly. 

B. Simulations — the use of the computer to 
model some aspect of reality or some set of real 



conditions so as to make the reality more amenable 
to manipulation and study. Simulations may cover 
a wide range ofphenomena, from planetary motion 
to airplane flight to presidential elections to the 
battle of Gettysburg. They encourage the user to 
come to understand the rules that are atplay in the 
model of reality that has been constructed. In 
theory, an understanding of the simplified model 
can lead to a better understanding of the more 
complex reality. Like pure games, simulations 
rarely teach anything in particular, but they en- 
courage problem solving and frequently stimulate 
many other kinds of learning. 

HI. Computer as Tool 

A. Special Purpose Tool — the use of tne com- 
puter to carry out a specific, narrowly-defined task, 
usually a significant task that is frequently repeat- 
ed. Tool programs of this kind are now beginning 
to proliferate because of their value in specific 
applications. For example, there are now numer- 
ous spellingprograms which, working together with 
a word processor (a general purpose tool) will check 
any piece of writing for spelling mistakes. 

B. General Purpose Tool — the use of the com- 
puter to assist people in carrying out a range of 
tasks within some general application area. Since 
the task is not specified ahead of time, it is up to the 
user to determine what is to be done and then to 
adapt the tool program to carry out that task. The 
power of general purpose tools is that the us er may 
repeatedly adapt and readapt them in virtually 
limitless ways for a multitude of purposes, thereby 
creating a kit of special purpose tools. Several 
types of programs fail into this category: word 
processors, database managers, spread sheet pro- 
grams, graphics utilities, music utilities, etc. 

C. Tool Making Tools — the use of the com- 
puter to create new tools of either a special or 
general purpose. Some programs are so broadly 
general in their scope that they have no immediate 
use except to create tools. All computer languages 
serve this function. In addition, there are more and 
more programs becoming available that are de- 
signed specifically to help people design useful 
tools without having to learn a computer language 
or become a sophisticated programmer. 

The Question Of Standards 

With a set of descriptive categories such as the one 
presented above, it is then possible to develop a 
set of standards appropriate for each category of 
software. And each set of standards will differ 
substantially from the set that is appropriate for 
another category. 
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For the more prevalent kinds of educational soft- 
ware — drill and practice and tutorial — some 
preliminary standards have begun to develop (e.g., 
programs should not make judgments about user 
behavior). The recent vast improvement in video 
arcade games has helped to establish some pre- 
liminary standards for educational games (e.g., 
programs must respond quickly and accurately to 
pser inputs). There are so few educational simu- 
lations and tool programs that we are just beginn- 
ing to be able to consider what quality programs in 
these categories might be (e.g., it seems clear that a 
functional and efficient "user interface" is one of 
the critical features). 

It is exceedingly important that any effort at soft- 
ware evaluation recognize that now is not the time 
for setting firm standards, but rather a time for 
developing flexible ones. We are at the very begin- 
ning of the application of computer technology to 
education. As noted before, software development 
is in its infancy, and so is hardware development. 
What was considered state-of-the-art a month ago 
is less than adequate next month... We are where 
the Wright brothers were at Kitty Hawk — just 
barely off the ground. 

We must work to facilitate the emergence of excel- 
lence in software, not restrain it. Whatever stan- 
dards we create must be flexible enough to em- 
brace the next stages of discovery and innovation. 
The standards we set for any kind of software 
today are based on the software we experienced 
yesterday. They will certainly have to be adjusted 
to do justice to the software that is produced 
tomorrow. 

The most effective way to keep in touch with the 
future is to maintain contact with groups working 
on software design and development. These groups, 
some in universities but most in private software 
companies, can be helpful in several ways: 

1) They frequently write orgivespeeches about 
their work prior to publishing it, and these accounts 
explain the specifications they have set and the 
reasons for them. 

2) . They are sometimes willing to share internal « 
documents or working papers that shed light on 
the standards they are setting for their work. 

3) They are often willing to share pre-publica- 
tion, test versions of their software for comments 
and reactions. Thus, there is an opportunity to 
sample new software well ahead of publication. . 

We must evolve a perspective on software evalua- 
tion that looks forward, not backward, thatasks for 
continuing improvements, rather than preserving 



established practices, and that challenges the in- 
dustry to give education the best hardware and 
software possible. 

Evaluation Projects — 
A Sampling 

Collected in this document are descriptions and 
some sample materials from a number of software 
evaluation projects. The following critical com- 
mentary ofseveral of these projects is presentedto 
clarify some of the serious dilemmas that are posed 
by trying to evaluate a new technology before that 
technology is well understood. I also wish to raise 
some questions about the appropriateness of the 
methodology used. 

MicroSIFT 

When MicroSIFT (Microcomputer Software and 
Information for Teachers) began operation in De- 
cember, 1979, hopes were high that this federally- 
funded software clearinghouse, housed at the 
Northwest Regional Educational Laboratory, 
would bring organization and clarity to software 
evaluation. It was the first attempt of its kind. In its 
first year, MicroSIFT established a national net- 
work of cooperating software evaluation sites 
where teachers and other educators could carry 
out systematic evaluations. Evaluation criteria 
were carefully developed and tested. The criteria 
were put on forms, which were then thoroughly 
field tested. Finally, the Evaluator's Guide was 
developed to assist people in the evaluation pro- 
cess. In its second year, MicroSIFT published its 
first evaluations. Software hi sen for evaluation 
represented a fairly narrow .,p. ;trum of the range 
of available software. Almost all of the evaluated 
programs used tutorial and/or drill and practice 
instructional techniques. I.n . ly opinion, the evalua- 
tions were ambiguous. But tlie differences between 
MicroSIFT ovaluations and my own evaluations of 
some of the same software make an important 
point: no one can rely entirely on evaluations per- 
formed by someone else. At best, these can be 
clues which may assist us in making our own in- 
formed ch< es, but they must never be substi- 
tutes for our own responsibility to evaluate. 

MicroSIF.T, as the first attempt of its kind, broke 
new ground and illuminated problems yet to be 
solved: first, evaluation by committee may be a 
worthwhile process if the committee can hammer 
out a consensus. But if the committee never re- 
solves differences and the consensus is created by 
nother.party, the result may be a very odd stew. 
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Second, while the goal of MicroSIFTs metho- 
dology — to attain reliability and credibility by 
reducing the level of subjectivity in evaluation — is 
a worthy one, the result does not always make clear 
distinctions among different programs or differen- 
tiate good programs from bad ones. In spite of its 
wrinkles. MicroSIFT has provided educators with 
one of the first evaluation forms to attempt to 
collect data in some depth. In addition, the crea- 
tion of a network of educates interested in eval- 
uating software and sharing their evaluations was 
valuable in itself, and contributed to the growing 
body of knowledge about evaluation as a process. 

EPIE 

Since 1967. EPIE (Educational Products Infor- 
mation Exchange) had been conducting evalua- 
tions of educational materials, equipment and sys- 
tems. In 198 1 . the EPIE Institute published one of 
the first major efforts to evaluate software (Micro- 
computer Courseware/Microprocessor Games). Thus, 
they brought to the task of evaluating this new 
technology many years of experience and a strong 
reputation in the field. In carrying out their soft- 
ware evaluations, they established a collaborative 
relationship with the Microcomputer Resource 
Center at Teachers College, Columbia University. 
Six programs are reviewed in EPIE's initial publi- 
cation, and, like MicroSIFT, the selection is all 
from tutorial or drill and practice materials. Once 
again, the reader may be left with the incorrect 
impression that the selection represents what is 
available. 

There is far less ambiguity in these reports. They 
reflect a stronger and surer point of view about 
what quality standards might be applied to educa- 
tional software, at least within the limited range of 
programs they considered (a brief introduction 
outlines the software attributes they considered 
and why). While the reports are undoubtedly the 
product of a group effort, they reflect stronger 
consensus and therefore convey a better sense of 
what the software is and what it is not. 

EPIE has recently joined forces with Consumers 
Union to publish both software (Microcomputer 
Courseware PRO/FILES) and hardware (Microcom- 
puter Hardware PRO/FILES) evaluations in a new 
"filecard" format. They have also formed the 
Consortium for Quality in Educational Computing 
Products which will publish a regular newsletter 
(MICROgram) as a forum for consumer concerns 
and issues. 

At this time, I have only one sample of their soft- 
ware evaluations to consider. Unfortunately, from 



this one sample, the standards that were set by 
EPIE's earlier efforts are not met in their new 
format or through their new methodology. The 
comments are briefer and made to sound more 
objective. A quantitative summary rating of major 
attributes is provided. There are brief quotes from 
magazine reviews and from student users, and 
there is one page of brief summary statements on 
various attributes of the program. While the for- 
mat appears more accessible and there is less 
reading required, the overall impression is now 
more like the MicroSIFT evaluations, that is, am- 
biguous and uncertain. And, sad to say, the reader 
does not know after .ading the evaluation whether 
the program is worth serious consideration. 

Courseware Report Card 

One of the most promising recent efforts to pro- 
vide software evaluation is Courseware Report 
Card, published by Educational Insights of Comp- 
ton, California. Interestingly, their reports have 
some of the good flavor of the early EPIE reports 
in that they present a reasonably coherent and 
consistent point of view. 

The company claims that it is "the first publication 
to offer a large volume of detailed critical reviews 
of educational software for p, variety of microcom- 
puter systems." They are probably right. Their 
first volume of reports consists of evaluations of 
twenty-two elementary programs and twenty-two 
secondary programs (four programs are included 
for both levels). While most of the programs re- 
viewed are tutorial or drill and practice, there are 
some games and a couple of simulations. The se- 
lection also includes programs published recently, 
suggesting that the authors are in touch with cur- 
rent software developments. 

The evaluation format is refreshingly straight- 
forward: a short introduction, a thorough descrip- 
tion, an evaluation on six criteria (performance, 
ease of use, error handling, appropriateness, do- 
cumentation, and educational value), a box of basic 
program information, and a box for a letter grade 
(A - F) on each of the six criteria. 

Three positive features of these evaluations stand 
ut. For the most part, when I have finished reading 
one of them, I know something about the value of 
the program. After reading several, I can distin- 
guish one evaluation from another. And, over the 
entire group, real distinctions are made: some pro- 
grams clearly emerge as very high quality, and 
others as low quality. For those that fall in between, 
the reasons are fairly clear. 

13 



Some Conclusions 

Education lives with the belief that good evalua- 
tion must somehow be objective. Both MicroSIFT 
andEPIE have made strongefforts to objectify the 
software evaluation process through elaborate 
evaluation forms and systems for digesting eval- 
uator responses. The result, as is often the case in 
educational evaluation, is a product that runs a risk 
of creating confusion. Courseware Report Card has 
no such process. 

In anew field, where the greatest need is for every- 
one to become much more literate, it is probably 
not a good" idea to have people looking to any sole 
source of authority. Rather, as I shall argue more 
fully below, the real need is for educators to be- 
come better informed so they can trust their own 
instinct and judgments. In our new computerized 
environment, we all face the responsibility of 
building our understanding through our own ex- 
periences, not through the processed experiences 
of others. 

The evaluation conducted by both MicroSIFT and 
EPIE depend i .part on the completion, by trained 
evaluators, of extensive evaluation forms. Though 
the forms do allow for some unstructured respon- 
ses, most responses are constrained by the ques- 
tions posed on the form. On the one hand, the 
questions are intended to guide the evaluator and 
produce some consistency across evaluators. But 
on the other hand, they also force the evaluator to 
view the software in predetermined ways that do 
not always fit the software being considered. For 
example, having reviewed these forms, I feel they 
show a strong bias toward the evaluation of tutorial 
or drill and practice software. Many of the ques- 
tions would not apply to a general purpose tool 
program, and the kinds of questions that would 
apply to this totally different type of software are 
not there. 

There are some other problems with any central- 
ized evaluation system. It is very slow at a time 
when advances in software development are being 
made very quickly. By the time a set of reports is. 
published, most of the software that has been eval- 
uated may have been improved or withdrawn from 
the market infavor of new and better products. It is 
no longer likely to be state-of-the-art. As tech- 
niques for software design improve, standards 
mustalso change. What was considered excellent a 
year ago may be only marginally acceptable today. 

Most software does not stay the same for very long 
because improvements are usually easy to incor- 
porate at any time. Software is constantly under- 
going improvements, based upon user reactions 



and review in publications. Both MicroSIFT and 
EPIE intend their evaluations to be of help to 
publishers in improving the quality of their pro- 
ducts. However, because of the inherent slowness 
of the evaluation process, it is possible that many 
of the criticisms made would be corrected by the 
time they reach the user. 

As software develops and as users have more ex- 
perience with a range of software types, we will 
come to understand more fully the vast potential of 
this technology for human improvement. Many 
serious thinkers have argued that computer use 
may well expand human learning beyond our cur- 
rent expectations, leading to a broadened and 
deepened human consciousness. We must be wary 
of software evaluation systems t!. r»t have the effect 
of restraining our vision of what is possible rather 
than extending it. 

In a fast-growing field, a reviewer has at least two 
major responsibilities: to maintain quality stan- 
dards, judging software in relation to those stan- 
dards, and to recognize and illuminate innovations 
that may well be the quality standards of tomorrow. 
It is hard to see how formal centralized evaluation 
systems can well serve either of these responsibil- 
ities. 

Some Alternatives to 
Existing Methods 

Probably the greatest need today is to facilitate 
the sharing of information about all of the various 
kinds of software and about programs that exem- 
plify each kind. We ought to be using the capacities 
of the computer and of advanced communications 
technology to help us with this task. We can build a 
software database that would include as much 
software information as possible from a wide vari- 
ety of sources. It could have several categories of 
information: general material on software and 
software evaluation, formal reports and software 
reviews, commentaries and critiques submitted by 
users, reactions and responses from publishers, 
etc. Some person or group could be assigned the 
task of setting up the database, maintaining it, and 
monitoring its use. 

If well done, such a database would provide a 
valuable information resource in a form that would 
be easily accessible by anyone looking for infor- 
mation on a particular issue or a particular piece of 
software. 

A database on microcomputer software, called 
RICE (Resources In Computer Education), has 
recently been established by the same group that 
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produced MicroSIFT at the North west Regional 
Educational Laboratory. It is reported to provide 
information on about 2,000 educational software 
programs. About 200 of these will have been eval- 
uated by the same network of cooperating institu- 
tions involved in the MicroSIFT evaluations. This 
database is currently accessible through the School 
Practices Information Network (SPIN), which is 
now being distributed by the Electronic Publishing 
Division of Scott, Foresman and Company. 
I look forward in the very near future to the time 
when it will be possible for educators wanting to 
share information about computer software ap- 
plications to communicate electronically with each 
other over a dedicated educational technology net- 
work. Such networks already exist in industry and 
higher education, and they provide users with an 
incredible resource for the sharing of ideas and 
information. Furthermore, the immediacy of the 
communication that is made possible from any- 
where in the country to anywhere else, can have a 
special impact on solving complex problems con- 
fronting people in remote places. 

Imagine, for example, that I'm a sixth grade science 
teacher doing a unit on the study of bones. I would 
like to know if educators on the networkhave either 
purchased or developed software themselves that 
might help me with my instruction. So, before I go 
home, I write a message for the network bulletin 
board and store it on a disk. Late in the evening, 
when the telecommunications rate are low, my 
message is placed on the network bulletin board. 
At various times the following day, people all over 
the country read my request, and a number of them 
send me responses. The followingmorning, I check 
my mailbox on the network and find to my delight 
that seven different commercial programs have 
been recomm ended for various aspects of teaching 
about bones, and two teachers have sent me pro- 
grams they have written themselves, which I can 
store on my disk and try out later. 

All the technology for such active sharing cur- 
rently exists, and I think it will not be long before 
such sharing will be common practice. 

One final point: good software takes advantage of 
all the features of computer technology — the 
interactive capacity, the speed, the graphic poten- 
tial (including animation), the sound potential (in- 
cluding voice), and the dramatic potential. Be- 
cause it is such a multi-dimensional medium, even 
the best evaluations of software fall considerably 
short of conveying to the potential user the nature 
of the software he or she wishes to know more 
about. With the advent of better telecommunica- 
tion via computers, software publishers will pro- 



bably offer demonstrations of the programs they 
are trying to sell, and software reviewers will be 
able to employ actual demonstrations of products 
in the course of their critical analyses. WINDOW, a 
new magazine in disk format, does exactly this 
type of software review. In time, this magazine may 
be available over the kind of educational network I 
have described here. 

What Can Be Done Right Now? 

All of this talk of iu aire possibilities may not sound 
very helpful to people in schools who are faced 
with having to make software decisions right now. 
Where can they look for immediate guidance until 
some better system comes along? 

Fortunately, there are a number of good sources of 
information on software. But, I would emphasize 
strongly that it is important always to take several 
sources of information into account in coming to 
any software decision. Probably the best sources 
are the educational computing magazines where 
software reviews for all types of computers are 
published regularly. Major computing trade mag- 
azines have been publishing special issues on edu- 
cation each year, and these are worth watching for 
because reviews of educational software are usually 
included (see Resource Information Section). 

Unfortunately, the same software is not usually 
available for every type of computer, though there 
is a growing tendency for publishers to produce 
programs for several computers. There are many 
good computer-specific publications that review 
educational software. For example, Peelings is an 
Apple-specific publication that publishes nothing 
but software reviews, many of which are of educa- 
tional programs. Apple's nonprofit Apple Educa- 
tion Foundation publishes The Apple Journal of 
Courseware Review, which includes extensive re- 
views of Apple software for education. 

Perhaps there will soon appear educational soft- 
ware review publications for other computers, 
which will fill in some of the gaps in software in- 
formation that currently exist. 

There are two distinct advantages to taking the 
time to become immersed in the software review 
literature. Obviously, the information will lead to 
better software choices. But, more important, 
through sampling the ways in which various peopl e 
approach the software revie process, one gra- 
dually develops a critical capacity of one's own. 
Classroom Computer News has emphasized this 
educational aspect of software reviews by provid- 
ing a review tutorial to supplement the reviews in 
each issue. InfoWorld periodically publishes its 
guidelines for software and hardware reviews to 
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inform the reader what it expects its reviewers to 
take into account in every review. Given the cur- 
rent stage in the growth of this technology, the 
strongest advice I can possibly give is for everyone 
to develop a critical capacity of his/her own. 

However, becoming conversant with the software 
review literature and developing a critical capacity 
is not enough. One must also spend time using 
software and attempting to apply, at least infor- 
mally, one's critical capacity. I am constantly as 
tounded by the number of educators I meet who 
seem to think that they can make intelligent edu- 
cational decisions about computer software with- 
out taking the time to learn how to use the tech- 
nology and become reasonably familiar with the 
software upon which the technology so completely 
depends. 

There is one major problem in the software pub- 
lishing industry that stands in the way of educators 
making informed choices of software: it is very 
difficult to get software on a preview basis or to 
return software that is found not to meet expecta- 
tions. Yet this is a consumer's right that educators 
must insist on exercising, esp3cially in a field 
where there is so much poor quality material. There 
is an encouraging trend now for publishers to 
change their position on this matter. I would urge 
all educators to help this trend by not purchasing 
from any company that does not permit them to 
evaluate software prior to purchase, or to return it 



after purchase (within a reasonable time period) if 
dissatisfied. 

One further note of warning. It is never sufficient 
only to evaluate a demonstration program or a 
salesperson's quick overview of how a program 
works. The actual software being considered for 
purchase must be thoroughly tried under favor- 
able conditions in order to be well evaluated. 

I repeat that computer software, if it is any good at 
all, is not like a textbook. The use of the computer 
in education can be (and should be) substantially 
different from the use of the textbook or workbook. 
And, furthermore, the use of the computer is also 
substantially different from the use of other audio- 
visual aids. No other technology has such potential 
for our work as educators. So when we talk about 
the need for computer literacy in our schools, we 
must start by accepting that the first task is for us 
to become reasonably literate ourselves. 

Then what's to be done? We have a professional 
responsibility to take every opportunity available 
to educate ourselves about computers and com- 
puter software so that we can make informed 
choices. It is not a responsibility that can be de- 
ferred to another time or referred to some outside 
agency. It is a critical responsibility that must be 
assumed right now by everyone involved in educa- 
tion so that our children will be able to function 
well in a changed and, I suspect, improved society, 
and so that our society can function well in a 
changed and, I hope, improved world. 



10 



16 



Approaches 
To Software 








12J 



Several useful approaches to evaluating educa- 
tional software have been developed during the 
last few years. In attempting to organize them, we 
have found that these approaches can be grouped 
along a continuum. At one end are those strategies 
designed to be used with large numbers of evalua- 
tors cooperating in an organized network under 
some given set of guidelines. At the other end of 
the continuumn are those approaches that provide 
discreet evaluation forms designed to be used 
freely by any individuals or groups at their own 
discretion. 

In the first category, we would place MicroSIFT 
and EPIE/Consumers Union (EPIE/CU). Briefly, 
MicroSIFT's system is made up of evaluators at 
sites around the country who use MicroSIFT's 
evaluation form and the Evaluat or !s Guide. Results 
are summarized by each site's computing expert 
and these summary reviews are published by Micro- 
SIFT; they are available from networkmembers or 
through RICE, a public access database. (The 
Evaluators Guide is now commercially available to 
educators who are not formal members of this 
network.) EPIE/CU has formed a .consortium of 
school districts around the country to evaluate 
both software and hardware. Evaluation special- 
ists at these sites, trained in using EPIE's evalua- 
tion form, supply EPIE/CU with their analyses, 
which are then published and available by sub- 
scription. 

Next to MicroSIFT and EPIE/CU on our con- 
tinuum, we would place School Micro ware Reviews 
andCoursewarc ReportCard. While not formalized 
into a network, these publications do have access 
to evaluators, either in-house or out, who supply 
them with evaluations according to given criteria. 
These evaluations are then published and avail- 
able by subscription. One notable difference in 



these approaches is that each provides publishers 
with the opportunity and the forum to respond to 
evaluations of their products. 

Third along our continuum, we place Minnesota 
Educational Computer Consortium (MECC), 
SOFTSWAP, and CONDUIT. The primary func- 
tion of these organizations is the development 
and/or distribution of software. Evaluation of their 
software is an essential ingredient to their respec- 
tive processes, but evaluation is not the primary 
purpose of their existence. MECC and SOFT- 
SWAP generally do not publish evaluations of 
their software. CONDUIT publishes evaluations 
in Pipeline, available by subscription. 

At the far end of our continuum — in the category 
of discrete forms designed to be used freely by 
individuals or groups at their own discretion —are 
the evaluation forms developed by the National 
Council of Teachers of Mathematics (NCTM), 
Scholastic, Inc., and Electronic Learning maga- 
zine. Each of these forms has supporting material. 
The form from NCTM is part of a book called 
Guidelines for Evaluating Computerized Instruction- 
al Materials; the Scholastic, Inc. form is part of an 
inservice training kit; and the form published in 
Electronic Learning is described as a synthesis of 
several forms. 

The information that follows includes an abstract 
and a copy of the evaluation form used in each 
approach. One minor note: excerpts from Course- 
ware Report Card and Guidelines for Evaluating 
Comptuerized Instructional Materials (NCTM) 
make reference to other portions of these publica- 
tions. The materials thus referenced will not be 
found here. Permission to reproduce the informa- 
tion in this section has been granted by each insti- 
tution or publisher included. 
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COURSEWARE EVALUATION 
PROCESS OVERVIEW 



microfSIFT 




300 S.W. Sixth Avenue Portland, Oregon 97204 (503) 248-6800 



The process described here was designed during the 1980-81 school year aa a framework 
for the evaluation of microcomputer-baaed instructional materials by the MicroSIFT 
clearinghouse. The components are a set of forms, the Evaluator 1 a Guide , and a network of 
educational institutions. 

The forms were based originally on the forms developed and used by the CONDUIT Project for 
evaluating computer-based instructional packages for post-secondary institutions. They were 
modified with additional concepts adopted from forma developed by the organizations and 
individuals. The "Courseware Description" form identifies the factual information necessary 
for evaluation and use of a package, including source, ability level, subject, mode of 
instruction, required hardware and software, instructional objectives and prerequisites. 
The "Courseware Evaluation" form is designed to be used after the information on the 
Description form is available. A copy of the rating portion is on the reverse of this page. 
In addition, it provides space for identifying major strengths and weaknesses, and suggestions 
for potential classroom uses. 

The Evaluator* s Guide is a book designed to be used by teachers and others who are evaluating 
courseware for MicroSIFT. It describes the use of the Description and Evaluation forms, and 
provides guidelines, suggestions and interpretations of each item on the Evaluation form. 

The microSIFT Network is a group of over 20 educational organisations serving elementary *nd 
secondary schools with computer services and other types of aupport. The network includes 
school districts, regional service centers, state departments and state consortia which have 
experience in serving local districts with inservice, software, computer time and services, 
curriculum materials and evaluation services. They have staff whose time is assigned to 
supporting the instructional computing activities of schools in their geographic area. 

The components above are used in the three stages of the process described below: 

1. Sifting - This is a first look at a package to determine that it is instructional 
in nature, will actually operate without problems on the appropriate microcomputer, 
and io complete with instructions. MicroSIFT staff complete this phase of the process. 

2. Description - A package passing stage 1 successfully is described in this stage using 
the Description form discussed above. The producer and MicroSIFT staff complete this 
stage for the most part. However, some information may be supplied in stage 3- 

3. Peer Review - Teachers with experience in the subject and grade or ability level of 
the material are selected from schools served by a network site to evaluate packages 
according to the Evaluation form and Evaluator 1 a Guide . A package is identified for 
a network site by MicroSIFT staff, and the teachers are selected by the instructional 
computing expert at the site. After the evaluationa are completed by the teachers, 
an evaluation is also done by the network site expert, who also completes a summary 
review encompassing all three evaluations. The summary review becomes the MicroSIFT 
evaluation of the package. 

Completion of the first three stages takes approximately three months. The resulting 
evaluations are professional opinions based on experience, and are not necessarily based 
on observation of student use of the packages. While some do include such use, the evsluators 
are volunteers, and their time does not always allow for extensive student involvement. Also, 
r package may be evaluated at a point in the school year not in conjunction with the time the 
topic is studied. 

A fourth stage of evaluation in greater depth is desirable for some packages because of their 
complexity or breadth of curriculum coverage. Such a stage might include pre- and post-testing, 
detailed observation of student activity while using a package, or other procedures. This 
stage is not being implemented by MicroSIFT at this time, although some approaches for it are 
being developed and investigated. 
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mkr^FT COURSEWARE EVALUATION 



!JX NORTHWEST REGIONAL 



EDUCATIONAL LABORATORY 



Package title 



Producer 



Evaluator name 
Date . 



Organization 



□ Check this box if this evaluation is based partly on your observation of student use of this package 



SA- Strongly Agree A-Agree D-Disagree SD • Strongly Disagree NA-Not applicable 
Please include comments on individual items on the reverse page. 
CONTENT CHARACTERISTICS 



(1) 


SA 


A 


D 


SD 


NA 


The content is accurate. 


(2) 


SA 


A 


D 


SD 


NA 


The content has educational value. 


(3) 


SA 


A 


D 


SD 


NA 


The content is free of race, ethnic, sex and other stereotypes. 


INSTRUCTIONAL CHARACTERISTICS 


(4) 


SA 


A 


D 


SD 


NA 


The purpose of the package is well defined. 


(5) 


SA 


A 


D 


SD 


NA 


The package achieves its defined purpose. 


(6) 


SA 


A 


D 


SD 


NA 


Presentation of content is clear and logical. 


(v; 


SA 


A 


D 


SD 


NA 


The level of difficulty is appropriate for the target audience. 


(8) 


SA 


A 


D 


SD 


NA 


Graphics/color/ sounci are used for appropriate instructional reasons. 


O) 


SA 


A 


D 


SD 


NA 


Use of the package is motivational. 


(10) 


SA 


A 


D 


SD 


NA 


The package effectively stimulates student creativity. 


(11) 


SA 


A 


D 


SD 


NA 


Feedback on student responses is effectively employed. 


(12) 


SA 


A 


D 


SD 


NA 


The learner controls the rate and sequence of presentation and review. 


(13) 


SA 


A 


D 


SD 


NA 


Instruction is integrated with previous student experience. 


(14) 


SA 


A 


D 


SD 


NA 


Learning can be generalized to an appropriate range of situations. 


TECHNICAL CHARACTERISTICS 


(15) 


SA 


A 


D 


SD 


NA 


The user support materials are comprehensive. 


(16) 


SA 


A 


D 


SD 


NA 


The user support materials are effective. 


(17) 


SA 


A 


D 


SD 


NA 


Information displays are effective. 


(18) 


SA 


A 


D 


SD 


NA 


Intended users can easily and independently operate the program. 


(19) 


SA 


A 


D 


SD 


NA 


Teachers can easily employ the package. 


(20) 


SA 


A 


D 


SD 


NA 


The program appropriately uses relevant computer capabilities. 


(21) 


SA 


A 


D 


SD 


NA 


The program is reliable in normal use. 



QUALITY 

Write a number from 1 (low) 
to 5 (high) which represents 
your judgement of the quality 
of the package in each 
division: 



Content 

Instructional 
Characterise cs 

Technical 
Characteristics 



RECOMMENDATIONS 

CH I highly recommend this 
package. 

d I would use or recommend 
use of this package with 
little or no change, (Note 
suggestions for effective 
use below.) 

□ I would use or recommend 
use of this package only if 
certain changes were made. 
(Note changes under weak- 
nesses or other comments.) 

CH I would not use or recom- 
mend this package, (Note 
reasons under weaknesses.) 



Describe the potential use of the package in classroom settings 



Estimate the amount of time a student would need to work 

with the package in order to achieve the objectives: 

(Can be total time, time per day, time range or other indicator.) 
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Strengths: 



Weaknesses: 



Other comments: 
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micr^FT COURSEWARE DESCRIPTION 




NORTHWEST REGIONAL 
EDUCATIONAL LABORATORY 



Title 



Version Evaluated 



Producer 



Cost 



Subject/Topics 

Grade Leval(s) (circle) pre-1 1 2 3 4 5 6 7 8 9 10 1 1 1 2 post-secondary 

Required Hardware 

Required Software 



Software protected? Dyes Qno Medium of Transfer: Dlape Cassette □ ROM Cartridge OS" Flexible Disk □ 8" Flexible Disk 



Back Up Policy 



Producer's field test data is available Don request Dwith package □ not available 



INSTRUCTIONAL PURPOSES & TECHNIQUES 
please check all applicable 

□ Remediation CDTutorial 

□ Standard instruction d Information retrieval 

□ Enrichment [ZlGame 

□ Assessment DSimulation 

Instructional OProblem Solving 

management \ — i _ , 

y UOther 
U Authoring 

CD Drill and practice 



DOCUMENTATION AVAILABLE 
circle P (program) S (supplementary material) 

P S Suggested grade/ability level(s) P S Teacher's information 



P S Instructional objectives 
P S Prerequisite skills or activities 
P S Sample program output 
P S Program operating instructions 
P S Pre-test 
P S Post-test 



P S Resource/reference information 

P S Student's instructions 

P S Student worksheets 

P S Textbook correlation 

P S Follow-up activities 

P S Other 



OBJECTIVES Dstated □ Inferred 



PREREQUISITES □ Stated □ Inferred 



Describe package CONTENT Ai\JD STRUCTURE, including record keeping and reporting functions 



Q ' ^>ck for more space 
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PO. Box 620 Stony Brook, N.Y 11790 
Nonprofit • Consumer Supported • Unbiased 
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MICROCOMPUTER COURSEWARE EVALUATION FACT SHEET 

The Educational Products Information Exchange (EPIE) Institute has, for many years, 
provided objective analyses of textbooks and other materials used in elementary and 
secondary schools. These analyses, used by numerous schools in the United States 
and Canada, have been an important part of the materials selection process. Now, 
with the advent of microcomputers in the classroom, it is essential that ^ process 
similar to that used in the analysis and selection of conventional classrcom materi- 
als also be applied to microcomputer courseware designed for classroom use. 

With partial foundation support, the EPIE Institute and the Microcomputer Resource 
Center (MRC) at Teachers College, Columbia University began a project in 1981 that 
systematically analyzed selected microcomputer courseware designed for school use. 
For that project, the EPIE Instituted instructional materials analysis instrument 
was adapted to the analysis of microcomputer courseware. This instrument systematiz- 
ed the analysis of courseware in terms of its instructional design - developer's 
rationale, learner objectives, content, methods and approach, and tests and means of 
evaluation. In the two years since the development of that first courseware evalua- 
tion instrument, EPIE has continued to evolve what it believes is an approach to 
courseware evaluation that is being responsive to the changing and evolving fieM of 
educational computing. 

In 1982, as the flow of microcomputer products being marketed for school and home 
use began to increase, EPIE, in addition to continuing its work with MRC, joined with 
Consumers Union of the U.S. C publisher of Consumer Reports ) and the consortium of 
school districts, including Albuquerque, Boston, Cincinnati, Housotn, Detroit, and 
Salt Lake City, to evaluate both microcomputer courseware and hardware products. EPIE 
evaluation specialists have provided extensive training in v sing the EPIE courseware 
evaluation form to teams of evaluators in these cities. Once certified as courseware 
analysts, members of these teams supply EPIE and Consumers Union with analyses of the 
courseware considered for purchase by school districts. As EPIE's courseware instru- 
mentation continues to evolve, chese trained specialists will use chese techniques. 

Using the resources of EPIE, Consumers Union, MRC and the six school districts, it has 
been possible to analyze many of the major courseware packages and hardware systems 
and an increasing number of smaller courseware packages. These product evaluations 
are available on a subscription basis to all school consumers in the form of EPIE and 
Consumers Union Courseware and Hardware PRO/ FILES (Product Files) and will be updated 
as new products are evaluated and new user evaluations are gathered on previously in- 
cluded products. Subscribers to EPIE-CU Micro-Courseware and Micro-Hardware PRO/FILES 
also become members of the EPIE-CU sponsored Consortium for Quality in Educational 
Computing Products and the International Council for Computing in Education (ICCE). 
ICCE publishes the consortium's newsletter, MICROGRAM, as part of its monthly publi- 
cation THE COMPUTING TEACHER. 

For further information about the consortium of school districts, the EPIE and Con- 
sumers Union Micro-Hardware and Micro-Courseware PRO/FILES and Evaluations, and other 
EPIE services, please call EPIE Institute at 212-678-3340 or 516-246-8664. 

EPIE Institute Executive Director: P* Kenneth Komoski 
Coordinator, Microcomputer Courseware Evaluation: Ellen R. Bialo 
MRC Direccor: James Dunne 
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fmrijhlfcfi MICROCOMP UJjyLJJJJJRJXHJULI evaluation form 

Analyst's Name: Date . 

plete Program Title : 



( name ) . __ Copyright: 

( addreSS > Authorisl: ZZZZIZ ZZZZZZZZ 

(phone) 

OWARE CONFIGURATION PROGRAM COMPONENTS PR1CE < 

Version of Firmware and/or 

rocomputer Model* Minimum K DOS needed Peripherals needed (specify number of disks t» nK > n <u nr 

ease star that microcomputer modeTTieTToT thlTifeflon ROM modll es; teacher ' s guide w/nwber of 

pages; Included support materials) 



»nded Users Specified by Pr oducer: *! pecify Pacing If : 
c ; ■ luuut - e ' .- components available separately 

je and/or Grade Range reduced costs for backup disks 

•ouplng (circle appropriate descriptors)) INDIVIDUALS PAIRS SMALL GROUPS CLASSROOM NETWORK 

:udent Entry Competencies (quote developer if possible, USE QUOTATION MARKS, infer if not specified) 



■iculum Role Specified by Producer: (circle appropriate descriptor(s) in each line) 
iTHEMATICS ' SPELLING LANGUAGE ARTS READING SOCIAL STUDIES SCIENCE 
IPPLEMENTAL BASIC PROGRAMMING LANGUAGE (COMPUTER) MANAGEMENT 

ILL and PRACTICE TUTORIAL EDUCATIONAL GAMING SIMULATION 

23 Copyright 1982 by EPIE Institute 



0™ ER (specify) 

0THER (specify) 

OTHER (specify) 
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CONTENT TOPICS : Stated or Inferred (circle one). If stated, 
USE QUOTATIOfTMARKS (e.g., "Adding Fractions"). 



Developer 's Rationale: Stated or Inferred (circle 
one). If "slaleCUST QUOTATION MARKS. 



Cite page number(s) : 



Development or Field Testing Evidence: Stated or Inferred 
(circle one). If stated, USE QUOTATTON MARKS. 



Cite page number(s) : 



Cite page number(s): 



Learner Objec tives: Stated or Inferred (circle one). If 
stated, USE QUOTATION MARKS. 



OTHER PROGRAMS ON DISK (if applicable) 



3 Examples: 



GOALS AND OBJECTIVES 



*Are goals and objectives supported by contents? YES 
Describe 

25 



NO Cite page number(s): 
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The rating scale (high, middle, low) is provided to assist you in making 
decisions about each question. Use the scale when appropriate, but always 
make remarks in the space provided, to clarify your response. 



CONTENTS 



* Describe what type of content is presented in what units (e.g., 10 games 
practice word attack skills at 3-5 difficulty levels). 



High Middle Low 



APPROPRIATENESS FOR INTENDED USERS 

* Is the match of content to student ability 
levels appropriate? 

Describe 



• Is the program scope reasonable (given 
program length and activities )? 

Describe 



* Is the readabi 1 ity (vocabulary, sentence 
structure) appropri ate? 

Describe 



•Is the tone of address appropriate? 
Describe 



• Describe uses of the program other than those suggested by the producer (e.g., 
special populations, grouping, individualization, etc.) 



ACCURACY AND FAIRNESS 

• Is the factual presentation accurate? YES NO 

Cite inaccuracies 
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C O N T E NTS (continued) 



o Is the program free of ^rrors in spelling , YES NO 

punctuation and grammar * 



Cite inaccuracies 



o Is the content social 1y bal anced? YES NO 

Describe 

o Is the content free of stereotypes (racial, YES NO 

sex-role, ethnicT^ 

Describe 



o Describe any potentially controversial content or methodology: 



CLARITY 

o Are directions clear? 12 3 

Describe 



5 Is display clear in terms of frame formatting 
and type readability ? 

Describe 



© Is display format consistent ? 12 3 
Describe 

© Is the use of examples and demonstrations 12 3 

effective? 

Describe 

© Is the use of cues and prompts effective? 12 3 

Describe 
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METHODS AND APPROACH 
TECHNICAL QUALITY 



©Is the software free of programing errors? 
Describe 


YES 


NO 




o Are control keys used consistently? 
Describe 


YES 


NO 




© Is the program easy to run? 
Descri be 


YES 


NO 



© Describe the terms of warranty: 



APPROACH TO CONTENT 

©Is the approach appropriate for intended users ? 12 3 

Describe 

* Does the approach enhance the presentation of 12 3 

the content? 

Describe 



DOCUMENTATION (Manual, Teacher's Guide, etc J 

• Are there instructional suggestions ? If so, YES NO 

describe their value: 



• Does the teacher's guide assist in organizi ng YES NO 

and relating the other instructionaTcomponents 
(workbooks, other materials) with the program? 



Describe 



Copyright 1982 by EPIE Institute 11/82 
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METHODS AND APPROACH (continued) 



© Does the teacher's guide provide for i ntegrating YES NO 

the program with the basic curriculum? 

Describe 

© Are the technical explanations for implementation YES NO 

clear and complete? 

Describe 

0 Does the producer recommend teacher training ? YES NO 

Describe 

o If the teacher training is not recommended, in 
your opinion, is it necessary? YES NO 

Of what should it consist? 



SUPPORT MATERIALS 

® List and identify role: 

© Are support materials essential for implementation YES NO 

of program? 

USER CONTROL 

e Are there opportunities (including menus) for the user YES NO 
to choose among content topics (e.g., lessons, games, 
etcTT? 

Describe 

9 Can the student review instructions ? YES NO 

Describe 

e Can the student exit the program at any time? YES NO 

Describe 



Copyright 1982 by EPIE Institute 11/82 
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METHODS AND APPROACH (continued) 



o Can the student alter the rate of presen tation YES NO 

(text rolling, problem display)? 



Describe 



Describe 





© Can the 
Describe 


student 


call on Help or Hint-type options? 


YES 


NO 












© Can the 
Describe 


teacher 


reset the Darameters of the program? 


YES 


NO 












o Can the 


teacher 


add or change content? 


YES 


NO 


Describe 







FEEDBACK 

e Is feedback appropriate for the intended users? 12 3 

Describe 



© Is feedback non-threatening ? 12 3 

Describe 



© Does feedback avoid reinforcing wrong response 12 3 

(as with an appealing graphic)? 



• Does feedback remediate ? 12 3 
Describe 

• Is feedback immediate ? YES NO 
Describe 



Copyright 1982 by EPIE Institute 11/82 
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METHODS AfJD APPROACH (continued) 
o Describe typical feedback after correct response: 



c Describe typical feedback after incorrect response: 



GRAPHICS 

© Are graphics appropriate for the intended users? 12 3 NONE 
Describe 

e Are graphics a program feature? 12 3 
Describe 

o Are graphics embedded in content? 12 3 
Describe 

• Do graphics add interest ? 12 3 
Describe 

o Do graphics avoid being distracting? 12 3 
Describe 

6 Are graphics clear ? 12 3 
Describe 

• Is there an option to use or not to use the graphics? YES NO 
Describe 

• Describe graphics (e.g., color, resolution, content, etc.): 
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METHODS AND APPROACH (continued) 



AUDIO 

©Is there an audio component to the program? YES 
Describe 

® Does audio enhance the program? 

Describe , 

o Can the audio be used with headsets and/or turned off? YES 

Describe 



RANDOM GENERATION 
© Is random. generation used in activities ? YES 
Describe 




5 Is random generation used in feedback? 
Describe 



IS THE PROGRAM EASY TO USE? 



YES 



NO 



EVALUATION 

*Does evaluation measure goals and objectives? 
Describe 



TESTS 

• Does program provide for overall placement? 
Describe 



t Are other tests included (lesson pretests, unit, 
; mastery, printed)? 

Describe 



YES 



NO 



YES NO N/A 



YES NO N/A 



Copyright' 1982 by EPIE Institute 

■/ 25 33 



11/82 



EVALUATION (continued) 



BRANCHING 

® Does program automatically branch for revie w? YES NO N/A 
Describe 

o Does program automatically branch after a lesson ? YES NO N/A 

Describe 



RECORDS MANAGEMENT 

o Is there a record keeping system? YES NO 

Describe 



© Is data stored for retrieval at any time ? YES NO 

Describe 



© Do student records identify specific difficulties ? YES NO 

Describe 

© Are student records cumulative? YES NO 
Describe 

e How many students in how many classes do records hold? 



© Do class records identify specific difficulties ? YES NO 

Describe 

© Are class records cumulative? YES NO 



Describe 
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EVALUATION (continued) 

© Is the management system easy to use? 
Describe 



° after'tesJs):^ 09 " 3 " 1 informs st udents Bf their progress (during/after lessons, 



SAMPLE FRAMES 

o Describe the location of 3-5 representative frames: 



DESCRIPTION AND REVIEW 

A. Describe the general structure of the program. 

1. How many lessons in a unit? On a disk? 

2. How sre lessons structured? 

a. number of i terns 

b. appropximate working time 

B * ?n S f^ b f h ° W 3 s * udent Progresses through a typical lesson from beginning 
to end (e.g., entry, contingencies, instructions, menus, tests, etc.). 

C ' ?p«ni b6 !5 e tea ' her ' s : o1e in the us e of the program (e.g., setting up a 
lesson, intervention, etc.). s y 

D. Discuss the educational value of the program (e.g., importance of content 
effectiveness of presentation, quality of instructional design). 

E. Documentation 

1. Describe the documentation (manual, teacher's guide etc ) 

2. Assess its usefulness technically. 

3. Assess its usefulness pedagogical ly. 

F. Management System 

1. Describe the tests and record-keeping in the management system. 

2. Assess the value of the tests. 

3. Assess the value of record keeping system. 

G. Assess the value of the support materials to the program. 
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SUMMARY 

H. Discuss whether the program takes full advantage of the unique capabilities 
of the computer. 

I. Describe the program strengths. 
J. Describe the program weaknesses. 

K. Make specific recommendations to the producer for revising and improving 
the program. 

L. What is your considered judgment of the overall quality of the program? 
CAPSULE RATING 

Rate each of the following on a scale of 1 (lowest) to 10 (highest). 

Overall rating of instructional design 

Overall rating of software design 
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INSTRUCTIONS FOR PARTICIPATING IN 
THE SCHOOL MI CROW ARE USER SOFTWARE REVIEW PROGRAM 



Here are the steps involved in having a product reviewed in 
SCHOOL MICROWARE REVIEWS : 

1. The software supplier (you) selects a product and sends 
it to Dresden Associates, P.O. Box 246, Dresden, ME 04342. 
Preferably, this should be a fairly new product and one 
which you think is especially good. 

2. Dresden Associates distributes software to evaluators 
based on their interests and capabilities. Evaluators sign 
an agreement not to copy the software made available. 

3. The evaluator reviews the software, evaluates it using 
Dresden Associates' questionnaire, and returns software and 
questionnaire to Dresden Associates. 

4. Dresden Associates sends the completed evaluation and a 
Courseware Description Form to the supplier (you) for 
comments and tc collect purchasing information, 
respectively. 

5. The supplier returns the comments and the completed 
Courseware Description Form to Dresden Associates. 

6. Dresden Associates prepares a review using the 
evaluator 's information and the information from the 
supplier. This is sent to both the evaluator and the 
supplier for final comment and approval. 

7. The evaluator and supplier return the review draft and 
comments to Dresden Associates. 

8. The review is published. 
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USER SOFTWARE REVIEW PROGRAM & 



; s ••S5en t i .1 * o 



'he Introduction. we 
■vinq -software quality 
'.no success of educa- 
and that the steps nec- 



ess^r *3z ^rinu;nq uoout this improve- 
ment ir** well <nown. However! taking 
inos'.» st-»p5 will require enormous re- 
sources. -iev* 1 ral aq«mcies are working on 
trie proc*em C ut -lave only scratcheo the 
surtace. 

This is -here SCHOOL MICROWARE PUBLICA- 
TIONS *nd its readers come in. Because 
tnere are so maTy cf us, we can accom- 
plish a really D ig iob with only a lim- 
ited amount of individual effort and ex- 
pense - and oet a lot in return! we in- 
vite you to oecome a member of the grow- 
ing network of educators all over the 
-.5. and Car. Jda who evaluate software for 
SMW REVIEWS. Here is what's involved: 

1. A prospective evaluator (you) obtains 
one or tore educational programs. (We 
have procra^s for review if you don't 
write or call as indicated below.) 

2. V ou evaluate zhe software using either 
tne form provided here (copy as neces- 
sary) or the Courseware Description and 
Courseware Evaluation forms used by the 
^lcroSIFT Project. (See page 1 for a 
description of MicroSIFT; their forms and 
supporting documentation are available 
for S2.00 from Dresden Associates,) You 
return the completed evaluation forms to 
Dresden Associates. 

3. Dresden Associates prepares a review 
from your evaluation and sends a draft 
copy to you and to th* software producer 
for comments. (We of co'irse reserve the 
right to accept or ro}ect individual 
evaluations, based on our assessment of 
their merits and the criteria stated be- 
low. ) 

4. You and the producer return the review 
draft and the comments to Dresden Asso- 
ciates. 



5. Dresden Associates incorporates the 
common ts in the review and publishes it. 

6. You receive copies of SHW REVIEWS. To 
compensate for the additional effort re- 
quired, you get more copies if you eval- 
uate large, mu 1 1 1 -prog ram products. 

The bottom line is that SWM readers can 
get inexpen s lve access to essential, in- 
depth evaluative information. 



ON SELECTING SOFTWARE FOP REVIEW 

First, we would prefer that programs be 
for the Apple, Atari, PET, TI-99/4, or 
the TRS-BO Model I/III. Second, a product 
sel ected for review should be either an 
independent, free-standing program or an 
entire system of int eg rated programs. 
However, it should not be only one member 
of an operationally integrated set. 

For example, several companies offer 
disk-oriented sets of programs to teach 
all aspects of a given broad topi c, e.g., 
dec imals . Prog rams may be included to 
pre-test ; to tutor several sub-topics 
such as add ition, conversi-n, and per- 
centage; to record and **^port perform- 
ance; and to coordinate '.ne operation of 
the entire set. These programs are inte- 
grated, not only by topic, but also in an 
operational sense. That is, it is diffi- 
cult or impossible to run one of them 
without having others available in the 
computer at the same ti me. It vou Id be 
desirable to review several programs of 
the set (on one or more forms), but not 
just a single program, as the latter does 
not give a complete and accurate picture 
of the entire product. 

On the other hand, most suppliers offer 
mul 1 1 pie -prog r&.n packages, which must be 
purchased as a single unit, but which 
consist of several programs which can and 
do operate independently, i.e., not more 
than one of them needs to be available 



for any given one to run. 
programs is a suitable 
evaluation* assuming that 
other criteria stated here. 



Each of these 
candidate for 
it meets the 



You have a better chance of getting your 
evaluation published if the product is 
not one that is likely to be reviewed for 
this project by several other persons. 
Evaluating a program two or three times 
is OK - in fact is preferred - but any 
more than that is probably a waste of 
time. We are not likely to publish more 
than two reviews of the same product un- 
less an additional one offers some very 
unusual and useful insights. Please con- 
tact us by mail or phone if you need 
guidance with regard to selecting a 
product to evaluate. 



THE EVALUATION FORM 

Our form has been used successfully by 
many persons but has not been validated. 
Therefore, it is not offered as a fin- 
ished product bu t as one which we can 
improve as we go along. Please feel free 
to make suggestions. 

Instructions for use of the form are in- 
cluded 11. it. Please take the time to 
accurately describe the product in the 
space for Functional Description, and be 
generous with comuents to support the 
numerical ratings you have assigned. If 
in doubt, take a look at some rev iews 
done by other evaluatora. We have found 
that it takes one to two hours to do a 
complete evaluation of a single program. 



SUBMITTING EVALUATONS 

Completed evaluation forms and other 
correspondence regarding review software 
should be Addressed to Barbara Courter, 
Research Coordinator, Dresden Associates, 
P.O. Box 246, Dresden, ME 04342 (Tel: 
207/737-4466) . 
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SCHOOL MICnuWAHIi KVALUAT ION KOHM „ :o ,,y A , ; NK| , nI , ()| 

Vour Nairn; .__Orua„ i z; , Uon .Position^ 

Add r o.'i n 

" ■ ■ 'IV 1 : _____ 

Prod Oct Nam.- Supplier PriC|1 \\°\ '<>~Q l r°~l*~. 

Subjects and i? radon .» ric« v Under rhin Na:no__ 

to Which Applicable K;it " ^t-udfrif. 

. __ — . , t t n\f > H»i<]uir<Hl 



PR ELIMINARY CONSIDERAT ION - 
in today's schools? y es 



°°M S thi ^ P r °9 r am contribute to the teaching of topTcT 

. e • „ ,-• w - No If No, give vour reason- * 

end of the form and omit twTSlanc of the questionnaire 



1 which should b<? tciuqht 
tor this answer i n the Comments section at the 



abLa to .the product, enter N/A. If the °ten ls unc e^r 'nte n' ° E T " * U ' " th ° itCm is not ap P' ic " 

ments at end or on extra sheets, qivinc Item nu m h"°, ' ent ° r U " Elaborate °" answers as necessary in com- 



giving item numbers. 



DOCUMENTATION 
program, e.g 
1. 



b. 



- List materials accompanying the 
teacher g guide, student workbook 
Indicate types of information included! 
Sugges ted course/sub ject, grade le vels . 

Goa Is. 

Performance objectives. 
Suggested teaching strateg ( ies > . 
Correlation with standard texts. 
Prerequisites for use of program. 
Student exercises, teacher answers. 
Operating instructions. 

and sample runs of program(s). 
description of the 



i . Lis t i 



ng 



j. If a simulation 
mod el u s ed . 

k. Suggested topics for follow-up 
d iscuss ions . 

1. Suggested references/activities fot 
follow-up. 

2. The documentation is written clearly. 

3. If a workbook i s included, the format 
and content are appropriate. 

INSTRUCTIONS GIVEN TO USER BY PROGRAM 

1. The instructions are adequate regarding- 
a. The instructional task to be performed. 



b. Details of how to interact with 
program. 

2. User has the option of skipping 
instructions if already known. 



the 



STUDENT -COMPUTER DIALOG 

1. Output is displayed screen by screen 
(paged) rather than scrolled. 
^2 . If output is paged: 

a. User has control over continuing to the 
next page. 

b. Amount of information in each page is 
appropriate. 

c The perceptual impact (amount of type 

and lines) is suitable. 

3. Output is spaced and formatted so as to be 
eas ily readable. 

COMMENTS - Please use 



4. Language is well suited to most students' 
read i ng ability. 

5. Uses correct grammar, jpelling , 
hyphenation and punctuation. 

6. Any grid or coordinate system used is 
consistent with common conventions. 

7. Students can respond with common symbols s 
ways of using them, e.g., right to left 
entry of sums . 

8 . Accepts abbrev ia t ions for common 
responses. 

9 . Provides for j nd i vidua 1 needr , e.g., 
opportunity to work with harder or easier 
material. 

10. Dialog is personalized, i.e., makes 
appropriate use of student names. 

11. Uses devices to get & maintain interest, 
e.g., variation of computer responses, 
humor, pace change, surprise. 

12. Makes good use of any special features 
compu te r : 

a. Graphics b. color c . Sound 



^3 .Reinforcing responses (indications of 
right, wrong, etc.) are appropriate. 

14. The number of wrong answers allowed is 
reasonable. 

15. Responds appropriately if allowed number 
of wrong answers is exceeded. 

16. Provides opportunity to get help if 
difficulty is encountered. 

17. Minimizes bad entries via devices such as 
objective formats (multiple choice, etc). 

18. Deals well with inappropriate entries, 
i«e., response to typing errors, etc., 
is intelligible and useful. 

19. Required entries are within students' 
capabilities (esp. typing, vocabulary). 

20. Reports student performance periodically 
and at end of session . 

MISCELLANEOUS CONCERNS 

1. If a simulation, the program gives a 
sufficiently accurate representation of 
the situation simulated. 

2. The concepts and vocabulary required to 
use the program are reasonable. 

3. Operates properly and is free of bugs. 

4. Is well structured and documented 
internally to facilitate any necessary 
debugging/modification . 



k i ,^ L tnis s P ace and additional sheets as necessary to provide anv information whirh » 
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COURSEWARE 

reportvcard 

ELEMENTARY 



Price: $12.50 



Evaluations of Microcomputer Programs for Education 



Volume 



September 1902 



No 1 



At last it's hero! — the premier issue of COURSE- 
WARE REPORT CARD, a software evaluation ser- 
vice for elementary and secondary educators. 

We initiated COURSEWARE RKPORT CARD 
after having become aware of the serious problems 
that exist concerning the selection of educational 
microcomputer software. There is a tremendous vol- 
ume of software to choose from with more programs 
appearing every month; for various reasons very 
little of tiiis material is available for previewing, and 
unfortunately a great deal of the product is not up to 
the standards teachers expect from more convention- 
al educational media. 

As for published evaluations, we found that many 
publications feature a few software reviews per issue, 
others carry reviews in greater quantity but on a rela- 
tively superficial and uncritical basis, while still 
others publish in-depth, analytical reviews in quan- 
tity but are connected with one or another of the 
major computer manufacturers and review only pro- 
grams for that particular make of computer. To our 
knowledge COURSEWARE REPORT CARD is the 
first publication to offer a large volume of detailed, 
critical reviews of educational software for a variety 
of microcomputer systems. 

Selection of Software for Review 

COURSEWARE REPORT CARD is reviewing pro- 
grams for Apple, Atari, PET/CBM, and Radio 
Shack TRS-80 microcomputers. This spring we 
examined the most recent catalogs of some 55 soft- 
ware producers and requested one or more programs 
from each for review. Most of the programs reviewed 
in this issue were sent to us in response to these re- 
quests. Others were received unsolicited from pub- 
lishers or were madeavailable to us by teachers, soft- 
ware dealers, or educational media centers. In our 
selection, we attempted to achieve a balance with re- 
gard to subject areas, grade levels, and hardware 
compatibility. 

With the exception of two teaching-aid programs, 
all the software reviewed in this issue is in the realm 
of computer-assisted instruction (CAI). Future 
issues will also include reviews of management pro- 



grains, authoring systems, and computer-literacy 
programs, including a survey of available versions 
of the 1X)G() and PILOT languages. 

Each COURSEWARE REPORT CARD review is 
self-contained and three -hole punched for notebook 
storage by subject area, grade level, or hardware 
compatibility. A complete list of programs reviewed 
in this issue can be found on the back of this package. 

Format for the Reviews 

Each review consists of an abstract briefly summa- 
rizing the nature and scope of the program, an over- 
all view of the program and its major features 
followed by a detailed description, and an evaluative 
section in which the program is examined critically 
in six areas: Performance, Ease of Use, Error 
Handling, Appropriateness, Documentation, and 
Educational Value. These categories are not ab- 
solute, and there is a great deal of overlap among 
them. In general, Performance refers to the pro- 
gram's overall content: electronic, verbal, and graph- 
ic. Ease of Use is concerned with use by both stu- 
dents and teachers relative to grade level and con- 
tent. Error Handling addresses the question of how 
the program reacts to input errors. Appropriateness 
refers to the ways in which the program makes use of 
the capabilities of the computer. Documentation 
covers the print-media instructional and support 
materials, and Educational Value is self-explana- 
tory (in management and teaching-aid programs, 
the heading Usefulness is substituted for this 
category). 

The capsule summary on the first page of each 
review that rates the program from "A" to "F" in 
each of these categories should be balanced against 
the more detailed evaluative material. In many 
cases a reviewer might have based a letter grade on 
criteria that a particular teacher might not find rele- 
vant to his or her needs. Views expressed in 
COURSEWARE REPORT CARD are to a great ex- 
tent subjective, and as educators our readers need 
not be reminded of the arbitrariness of grades. In 
general, each reviewer has full discretion in as- 
signing grades, with one exception being in the 
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Appropriateness category. It is our contention that 
drill and practice is not a particularly appropriate 
use of computers in education, and it is our policy not 
to assign any drill-and-practice program a grade 
higher than a "C" in this category unless it is en- 
hanced by additional features. 

About Our Reviewers 

Any unsigned review appearing in COURSEWARE 
REPORT CARD was prepared by one or more mem- 
bers of our editorial staff: Mark Falstein, Laura 
Cohen, Terry Garnholz, Emily M. Hutchinson, 
La Vonne Miller-Casey, and Marcia Shank. 
COURSEWARE REPORT CARD is a publication of 
Educational Insights, a long-established publisher 
of supplementary educational materials. Every 
editor on the staff is a former classroom teacher and 
has had extensive experience in the evaluation, de- 
velopment, and production of curriculum materials 
for a wide variety of subject areas, grade levels, and 
applications. Managing Editor Mark Falstein is the 
author of The Computer Is Here, a computer-literacy 
activity kit for children and has served as editor of 
The Electronic Classroom, a newsletter on computer 
applications in education. 

The following editors also contributed material to 
this issue of COURSEWARE REPORT CARD 
(Secondary edition only): 

Linda Marie Hary chairs the mathematics depart- 
ment at John Glenn High School, Norwalk, Cali- 
fornia, and is a member of the Computer In- 
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struction Committee of the Norwalk-La 
Mirada School District. 
Terry Humphries directed the implementation of 
microcomputer-based education in the Bell- 
flower Unified School District, Bellflower, Cali- 
fornia. He has taught programming and compu- 
ter literacy to administrators, teachers, parents, 
and students; and served as a computer consult- 
ant to other school districts in southern 
California. 

Laurence Johnson, a math teacher at South 
Pasadena (California) High School, has used 
computers extensively in his classes for the past 
three years and is currently working under a dis- 
trict grant to develop a computer-based mathe- 
matics curriculum. 

Any reader who feels that he or she is qualified to 
become a contributor to COURSEWARE REPORT 
CARD is encouraged to send us a resume and an ap- 
propriate writing sample. We're particularly in need 
of reviewers of software for the PET/CBM and TRS- 
80 systems. We will pay 0 for each review we 
accept for publication. Unsolicited manuscripts are 
not encouraged, but will be considered if written ac- 
cording to our format and specifications and 
accompanied by a self-addressed stamped envelope. 
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Complaint Department 

Any software publisher who feels we didn't give his 
or her product a fair evaluation is in vited to submit a 
letter of reply. We will publish all such letters that 
are limited to 200 words in length. 

Teachers and administrators who have used pro- 
grams reviewed in COURSEWARE REPORT CARD 
and who would like to offer corroborating or dissent- 
ing opinions are also encouraged to write to us. 
Please limit your replies to 100 words. 

We'd like to know what the educational computing 
community in general (producers as well as con- 
sumers) thinks about our new publication. Write, or if 
you're attending EdCom in Los Angeles October 
21-24, stop by our booth. We'll be the one flanked by 
the four big bodyguards trained in kung-fu. 
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WHAT IS MECC? 

The Minnesota Educational Computing Consortium is a public organization 
established by the State government to assist Minnesota schools and colleges 
implement educational computing. MECC was established in 1973 and began pro- 
viding direct services to educators in 1974. Three types of services support 
the use of computers in instruccion: 1) providing low-cost access to computing 
resources including the operation of a statewide timeshare system and the 
establishment of microcomputer purchase contracts; 2) development of educational 
courseware; and 3) inservice training of educators. MECC's knowledge and 
expertise in the educational computing field comes from nearly a decade of 
working with and providing leadership for hundreds of local educators on a 
daily basis. 

Contact MECC at 2520 Broadway Dri\/e, St. Paul, MN 55113-5199, 612/638-0600. 

Note about the following forms: "Computer Courseware Review Form" is most useful 
for discussion and comparison of software, perhaps by a committee- It may be used as 
a means of discarding obviously poor software. The "Microcomputer Educational Materials 
Evaluation" and "Student Evaluation" are more than checklists; they may be used by 
teachers actually using software in the classroom, and provide room for extensive 
notation. 
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MINNESOTA EDUCATIONAL COMPUTING CONSORTIUM 



Instructional Systems Development 
2520 Broadway Drive 
St. Paul, Minnesota 55113 



MICROCOMPUTER EDUCATIONAL MATERIALS 
EVALUATION 



Reviewer s Name- 



Review Period From: 
To: 



School/District 



mIcPo W,M USe y ° Ur classr00m v aluation of the enclosed educational materials as a basis for a review to be published in the 
USERS newsletter. In your evaluation, consider the total package— software and support material. 



Title of Package:, 



List the programs from the package that you trialed with the students: 



Classroom Review Situation 

Please descibe the number and ability levels of students who used the program. Describe the situation in which it was trialed. 



I. Ease Of Use 



Can students operate the program easily? Are directions clear? Is it easy for students to start the programs? end the 
programs? Is requested input easily understood? 



April/May/June 1982 MONITOR 
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II. Level of Student Interest 

Did the students who tried the programs finish without your prompting? Did you fee! that they were interested in and 
motivated by the materials? Did students want to do the activities again? 



III. Support of Teaching Process 

Was the material easily integrated with classwork? Did you feel it was a valuable instructional tool for communicating its 
stated objectives? 



IV. Use of Microcomputer Capabilities 

Was the interactivity of the microcomputer used to its full extent: Was the student addressed by name? Was feedback on 
student responses immediate and specific? Did students have control over rate or level or sequencing of presentation? Did 
students receive information on how well they were doing? 



Were graphics, sound, color used effectively? 



April/May/ June 1982 MONITOR 
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V. Documentation 

^eot h y e p.ng? POrt d0Cumentatl0n clear? ...complete? ...useful? Did it identify prerequisite learnings? Was it free of 



VI Technical Accuracy 

Did you have any technical problems with the program? 



VII. Educational Content or Value 

Su'^ ^^...appropriate for the audience? Was the quaN.y of 



Summary 

This program is educationally sound, and I would recommend it to other schools. 

strongly disagree : : strongly agree 

Summary Comments 
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M.E.C.C. 
STUDENT EVALUATION OF 
MICROCOMPUTER MATERIALS 



STUDENT'S NAME 

SCHOOl 

DATE 



GRADE 



TEACHER. 



PACKAGE TITLE. 



PROGRAM NAME 



. APPROXIMATE AMOUNT OF 
TIME SPENT 



CIRCLE THE BEST ANSWER: 
(Write comments if you want) 

Using this lesson was: 



Comments: 



1. Easy 

2. Somewhat hard 

3. Very hard 



After doing this lesson I would: 



Comments: 



1. Like to do another like this 

2. Rather not do any more 
lessons like this 



From doing this lesson I learned: 



Comments: 



1. A lot about the subject 

2. A little bit about the subject 

3. Nothing 

Describe what you would do to make this lesson better. 



Thank you for your help! 



$1981 Minnesota Educational Computing Consortium 
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Subject Area 
Grade Level: 



Specific Topics: 



MINNESOTA EDUCATIONAL COMPUTING CONSORTIUM 
COMPUTER COURSEWARE REVIEW FORM 



Package Name: 
Producer: 



Reviewer's Name 
School 



Classroom Tested: 
Required Hardware: 



YES 



Check ail applicable items: 



Instructional Purpose 

Remediation 

Standard Instruction 

Enrichment 
Support Material 

Available 

Not Available 

Slsa of I na tra cU u na l Group 
Individual 

Small Group (up to 4) 
Large Group (4+) 



Describe Classroom Review Situation: 



# of programs 
Cost $ 



Date 



NO 



Instructional Technique 

Drill and Practice 

Tutorial 

Information Retrieval 

Game 

Simulation 

Problem Solving 

UtiUtv 

Other: 



Add comments to the categories evaluated on the back: 
Educational Quality: 



Student's Reaction: 



Support Materials: 



Technical Quality: 



Overall Reaction: 



This program is educationally sound and I would recommend it to other teachers. 



Strongly 
Disagree 



Stronglv 
Agree 



©1981 Minnesota Educational Computing Consortium 
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COURSEWARE EVALUATION FORM - PAGE 2 



EDUCATIONAL CONSIDERATIONS 



Do the Instructional Objectives match those of the district 



curriculum projects? 


YES 


NO 


Are the objectives ciear? 


YES 


NO 


Are the concepts presented logical? 
Is the information accurate? 


YES 


NO 


YES 


NO 


Is it of appropriate difficulty level? 


YES 


NO 


Is it sound theoretically (based on Learning Theory)? 


YES 


NO 


Is is appropriate use of the computer? 


YES 


NO 


Has it been field tested? 


YES 


NO 


Is it at the appropriate reading level? 


YES 


NO 


Can instructors adapt package to their objectives? 


YES 


NO 



CLASSROOM 



Are objectives defined to student? YES NO 

Is it easy to use? YES NO 

Are the directions clear? YES NO 

Is the feedback effective? YES NO 

Are the displays effective? YES NO 

Are students provided information on how well they are doing? YES NO 

Is it easy to implement? YES NO 



SUPPORT MATERIAL 

Are instructional objectives defined? YES NO 

Is it necessary to use the program? YES NO 

Does it jive teacher background information necessary to ooerate? YES NO 

Does it provide student material? YES NO 

Does it tell teacher how to incorporate in instruction? YES NO 



TECHNICAL QUALITIES 

Is the text grammatically correct? YES NO 

Is the text or the screen readable, (uncluttered* without 

scrolling)? YES NO 

Is adequate time given to read the text? YES NO 

Are grapnics, color, and sound used effectively? YES NO 

Are the questions clear? YES NO 

Is inDut analyzed effectively? YES NO 



ADDITIONAL QUALITIES - (Check all that apply) 

Drill and Practice/ Game 

program provides difficulty levels 

i additional practice provided on 
proDiems tacking mastery 

record of student performance 

jset* tn control of proDiems 

feedback 

positive 
negative 
_^ provides hints 
i tutorial 

Tutorial 



evaluation included in learning 
student is an active learner 
sequence approDriate 
segments can oe accessed by 

learner 
feedback 

_ student on wrong resDonse 

orovides clues 

reinforces learning 



Problem Solving/Simulations 
segments accessed out of 

seouence 
provides hints, clues 
enables students to generalize 
provides graDhic 

reoresentation 
identifies assumptions 

(if simulation) 
model valid 



Utility /Information Retrieval 

allows for maximum 

flexibility 
low degree of comDlexttv 

for user 
orovides necessarv options 

to achieve ODjectives 
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How to 
Start a 
Software 

Exchange 

by Ann Lathrop 
and Bobby Goodson 

"What is different about 
SOFTSWAP? Isn't it just 
another users' group?" 

Recipe fora ^ eX perienced 
1 CATORS, FF1CE OF 

bers, oUS loans of 

CompuC° 101 ' i 
MiPnts With one i 

1 compute . tnte with 

Ut marinate for abo p 
ComP uiex 

where'. — — 



Lathrop is Library Coordinator at 
the San Mateo County Office of Educa- 
tion. Redwood City, CA. 

Bobby Goodson is President of Computer- 
Using Educators, and Computer Resource 
Specialist at the Cupertino Union School 
District, Cupertino. CA. 




Visitors to Softswap receive assistance from Microcomputer Center workers and from one another 



■he Microcomputer Center in Red- 
wood City, California, is a gathering 
place for members of Computer- 
Using Educators (CUE) and visitors from 
other areas who enjoy sharing their ideas, 
problems and expertise in this unique re- 
source center. It's an exciting place to 
spend an afternoon, a few days, or longer, 
and the welcome mat is always out for 
educators who want to help with, various 
CUE projects, to copy SOFTSWAP pro- 
grams, or who simply like to sit and talk 
with other microcomputer enthusiasts 
who happen to drop in. Questions and 
answers fly back and forth, and frequent 
arguments as to which answer is correct 
reflect the diversity of opinion and ex- 
perience represented by the group. At 
any one time the questions may vary 
from "What computer is best?" and "Can 
1 do the school budget and attendance on 
this one?*' to "How can I get this pro- 
gram to run off of our Corvus? >l and 
"What is the difference between 1 2 and 
3.3 DOS?" 

Many answers are derived from do- 
it-yourself experience with the micro- 
computer systems, commercial packages, 
and SOFTSWAP programs which are 
available to visitors. Other answers are 
found by consulting the resource file of 
names and addresses of people experi- 
enced in almost all aspects of microcom- 
puting. The books, journals and extensive 
reference files in the library provide still 
i more answers to visitors' questions. The 
people in the Center, both staff and visi- 
tors, are also a valuable source of infor- 
mation and expertise. 

The Microcomputer Center was 
established just over a year ago in the Li- 
brary of the San Mateo Educational Re- 
sources Center (SMERC) as a joint 
project of Computer- Using Educators 
and the San Mateo County Office of Edu- 
cation. Visitors have come from through- 
out California, from many other states 



and Canadian provinces, and from 
Australia and France. All have been in- 
terested in learning more about the 
challenging potential of microcomputers 
as new instructional media. By support- 
ing the establishment of this unusual edu- 
cational resource, the San Mateo County 
Office of Education, under the leadership 
of Superintendent William K. Jennings, 
continues its traditional role of develop- 
ing innovative educational programs in 
California. 

There has been no outside funding of 
the Microcomputer Center. The San 
Mateo County Office of Education pro- 
vides the space, maintenance and part- 
time staff support a.s a service to educa- 
tors in the coin. \ and throughout the 
San Francisco H.iy area. The active in- 
volvement of Computer-Using Educators 
as co -sponsor of the Center has been the 
other major factor in it r success. 

CUE has grown into an organization 
of over 1600 educators, with members 
in 40 states, 12 provinces and 12 foreign 
countries. CUE members donate hun- 
dreds of hours of volunteer time to or- 
ganize and participate in activities of the 
center, including work on SOFTSWAP 
programs, demonstrations of new equip- 
ment and software, and the commercial 
software evaluation project. 

Many members of the county office 
staff cooperated in the development of 
the Microcomputer Center and continue 
to be active in its operation. The original 
concept was presented by Ann Lathrop, 
Library Coordinator, who continues to 
supervise the SOFTSWAP and the soft- 
ware evaluation project. Assistance and 
support for the project was given by Dr. 
Curtis May, Director of Library Services, 
whose commitment has been essential to 
the successful establishment of the cen- 
ter, Visitors receive friendly help from 
Janice Marshall, Library Assistant, and 
from her staff of student aides. Curricu- 
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tributcd software could be reduced or 
eliminated if donors would check the fol- 
lowing: 

1) Does the program run with relatively 
f e w p roblem s? (Unto rt u n a t e 1 y . w e 
receive many nrograms which do not 
load/run at all.) 

2) Does the program trap input errors? 
Does it crash if the user inputs nu- 
meric responses when the computer 
is expecting alpha, or vice versa? 
What happens if the user hits an 
unwanted PINTER/ RFTU RN? When 
there is an input error, is the com- 
puter response clear ami simple? 
"Input error, redo from the start" 
is not helpful to the novice. 

3) Are all the words spelled correctly? 

4) Arc the instructions and other infor- 
mation easy to read on the screen? 
Is it formatted to avoid split words at 
the end of a line? \\ is best to avoid 
crowding and to clear the screen at 
appropriate internals. Instructions 
should be written >'t a reading level 
appropriate for the intended user. 

5) Is the positive reinforcement for the 
correct responses mon interesting 
that the negative reinforcement for 
errors? Graphics in most "Hangman" 
type games, for example, are usually 
more exciting when the user loses 
and is "hanged." Students will often 
deliberately miss a problem or a 
word in order to see a rocket ex- 
plode, a boat sink, etc. It is best to 
try to provide strong encouragement 
for doing a task correctly and yet 
avoid over- praising. It is also better 
to provide several responses for both 
correct and incorrect answers - a 
little variety adds a great deal to the 
program. 

6) Have you included REM statements 
as needed? They make it easier to 
use a program effectively. It is espe- 
cially helpful to include statements 
telling the teacher how to modify 
word lists, problems or other entries. 




Bobby Goodson, one of the moving forces of 
Computer-Using Educators, possesses a wealth 
of information about educational software* 



7) Do you make negative remarks to or 
about the student? Sarcastic state- 
ments can be discouraging. We re- 
move comments like "Boy was that 
dumb," "Stupid," "Dummy, " etc. 

8) Do you provide hints atul/or even- 
tually, give the user the correct an- 
swer'? We usually allow only 3 or 4 
incorrect responses before providing 
either a hint or the correct response, 
often with some instruction of ex- 
planation. Exceptions are programs 
which are tests or timed drills for 
accuracy. 

{ )) Is the content accurate? Are the defi- 
nitions and/or explanations simple 
and straightforward? Do they corre- 
spond to the material in the more 
frequently used texts? 
10) If it is a drill-and-practice program, 
do examples come up in random or- 
der? Are there enough problems or 
questions that the student will not 
have all the same examples if they 
run the program again? If it is appro- 
priate, does the student have the op- 
tion to return to the beginning of the 
exercise and try it again? 
Finally, we want to be sure that the 
program su bin it ted is truly an original 
creation of the author. If it is an adapta- 
tion from another program, from a book 
or a journal we want to be aware of that 
fact. We will request the author's permis- 
sion to include the program in our col- 
lection. If we discover that a SOFTSWAP 
program is actually an infringement on 
someone else's copyright, we remove it 
from the dissemination disk. 

The most exciting programs we re- 
ceive aie the creative ones that do more 
than just put workbook pages and drill 
problems on the screen. Contributed 
so ft w a re is i m p rov ing as ed u ca t o rs b c- 
come more sophisticated users and pro- 
grammers. Newer programs are often 
more challenging to the student user and 
make better use of the unique capabilities 
of the computer. We are especially 
pleased when we receive donations- of 
this type and take special pride in being 
able to make them available to other 
teachers through the SOFTSWAP. 

Evaluation of Commercial Software 

The newest project in the Microcom- 
puter Center is the development of an ex- 
emplary collection of commercially 
produced educational software. We have 
contacted publishers of microcomputer 
software with requests for review copies 
of their educational packages. To date the 
response has been very encouraging. 
Many publishers appear to welcome the 
opportunity to put their materials on dis- 
play for evaluation. CUE members are 
previewing and evaluating this software 
during the summer and the project will 
continue throughout the coming school 
year, with tentative plans to publish a col- 
lection of reviews. Visitors to the center 



may preview the software and are invited 
tt> help with the review process. However, 
this software may not he copied. 

Our goal is to make selected software 
available for each microcomputer system 
in the center in order to demonstrate the 
special features of the system We also 
want to give the potential user Some idea 
of the range and quality of materials 
being published in various subject areas. 
Finally, we will provide the opportunity 
for educators to preview a wide variety of 
Software before making purchase deci- 
sions. Since it is currently very difficult 
to find reliable critical reviews of comput- 
er software, and because many teachers 
prefer to make their own evaluation be- 
fore purchase, we believe that this will be 
an especially valuable service. 

We are also developing a collection of 
software to demonstrate administrative 
applications of microcomputers. We will 
preview and display software designed for 
school administrators, including word 
processors and data base management 
programs. We are also especially interest- 
ed in potential uses of this type of soft- 
ware in special education programs. 

As part of our commitment to en- 
couraging the development of good criti- 
cal evaluations of microcomputer soft- 
ware, the Microcomputer Center is a 
participating member of MicroSIFT, a 
Northwest Regional Educational Labora- 
tory project funded by the National In- 
stitute of Education. The objective is to 
produce and disseminate a high-quality 
evaluation instrument and to publish re- 
views of computer software evaluated in 
terms of the criteria developed. CUE 
members', under the coordination of the 
Microcomputer Center staff, have already 
completed the first round of MicroSIFT 
program reviews. 

Summary 

Any new project is accompanied by 
often unexpected and urgent needs. More 
than the usual number of critical needs 
seem to surface with an educational 
microcomputer project — perhaps because 
there is apt to be a lack of readily avail- 
able support resources. The partnership 
of CUE and the San Mateo County Office 
of Education has created the Microcom- 
puter Center to meet many of these 
needs. We have provided educators with a., 
place to see and try various microcom- 
puter systems where other educators who 
understand their concerns are available. 
We have created a demonstration site for 
commercial software, where teachers can 
"try before they buy," and where they 
can develop the ability to do critical and 
objective evaluation of software. We are 
providing free, educational software and a 
way for teachers to exchange their own 
programs with others. We hope we have 
created a model that will work in other 
areas and thus will play an even more sig- 
nificant role in developing uses of micro- 
computers in education,, U 
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lum development programs and in-service 
training are the responsibility of LeRoy 
Finkel, computer consultant, and of 
Walter Smithey, math-science consultant, 
working under the direction of Tom 
Quinn, Director of Curriculum. Dr. James 
Fee, Director of Media Services, offers 
assistance with hardware applications and 
demonstrations. An extensive program to 
interface computers and microcomputers 
throughout the county's schools is being 
developed by Dr. Al Grossman, Admini- 
strator of the Office of Information Ser- 
vices. The entire project is under the ad- 
ministration of Dr. Don Halverson, As- 
sistant Superintendent for Educational 
Support and Planning. This team ap- 
proach by the county otTicc staff, 
coupled with the expertise and commit- 
ment of Computer-Using Educators, has 
been crucial to the successful develop- 
ment of the Microcomputer Center. 

M a n u fa e tu re rs of m i c ro co m p u te r 
hardware have also been generous in their 
support of the center. Tandy (Radio 
Shack) and Commodore (PET) provided 
the first microcomputer systems and thus 
initiated the support that has made pos- 
sible all subsequent development. Each 
Firm set up a complete system with mi- 
crocomputer, duaJ-disk drives, printer 
and cassette- recorder, i?ll on long-term 
loan of the Microcomputer Center. A li- 
brary of commercial programs loaned to 
the center by each firm includes educa- 
tional software and word processing sys- 
tems. 

Other firms providing equipment for 
the center including APPLE, Atari, Bell & 
Howell and Compucolor. All of the sys- 
tems are on long-term loan; the center 
has not purchased any hardware. CUE 
members were of great assistance in con- 
tacting the various firms* and securing 
their support and cooperation. 

All of the manufacturers have been 
helpful in maintaining the equipment in 
good condition, despite heavy use in the 
center. They also provide new models as 
they are developed. This strong commit- 
ment on the part of the manufacturers 
has been another critical factor in the suc- 
cess of the center. 



and also, supervised the processing of 
SOFTSWAP programs for each computer. 
The committee consisted of Jim Love 
(PET), Brian Sakai (TRS-80), Dave Stone 
and Bob Enenstein (APPLE), Marge 
Fitting (Compucolor), and Pat Tubbs 
(Atari). The chairpersons also conducted 
informal demonstrations and evaluation 
sessions during the summer. As we enter 
our second year of the SOFTSWAP 
project, Leslie Grimm is the new chair- 
person for APPLE; the other systems con- 
tinue under the direction of the original 
Advisory Committee members. 

Public support of the SOFTSWAP is 
encouraging. A | arg e number of volun- 
teers continue to donate programs, time 
and expertise to this growing collection. 
As new programs flow into the center 
they arc routed to the CUE volunteers 
for evaluation, debugging and processing. 
Finished programs are placed onto 
SOFTSWAP dissemination disks which 
can be copied without charge by anyone 
who visits the center. 

The large number of requests from 
outside the San Francisco Bay area led to 
the mail-order policies adopted this 
spring. The first catalog was published in 
the March 1981 CUF Newsletter and 
lined -v er 200 programs on 12 disks. 
E<- '^nal Jisks have been complet- 

ea \ total number of programs 

in the bOt TSWAP to nearly 250. Anyone 
who sends an original program on a disk- 
ette as a contribution to the SOFTSWAP 
can request a free disk of programs in ex- 
change. Disks are also sold for the cost of 
the disk, handling and postage. During 
the first three months of operation over 
3 70 disks have been sent out, some going 
as far away as Buenos Aires, Singapore 
and Paris. 



SOFTSWAP Procedures 



Development of the SOFTSWAP 

The SOFTSWAP began as an ex- 
change of instructional programs at the 
Asilomar Math Conference in 1979. 
Vinec Contreras, San Jose State Universi- 
ty professor, organized this first attempt. 
The programs contributed by participants 
were deposited in the SMERC Library in 
the spring of 1980. The first of the equip- 
ment being loaned to the center arrived 
from the manufacturers just as school was 
out. Interested educators worked through- 
out the summer to evaluate, edit and re- 
vise the programs from the Asilomar Con- 
ference. 

CUE members served as an Advisory 
Committee for the Microeompu ter Center 



What is different about the SOFT- 
SWAP? Isn't it just another users' group? 
It is a users' group, but the primary dis- 
tinction is one of focus - SOFTSWAP 
concentrates on the development of edu- 
cational software. Most programs are 
short, .stand-alone instructional units. 
Many are drill- and- practice exercises 
written for the elementary school or for 
remedial work at the secondary level. 
About one-third of the programs are 
math oriented. Most games have been ex- 
cluded since these are generally available 
from other sources. Only a few utilities, 
those of special interest to teachers, were 
on the first series of SOFTSWAP disks; 
these also are readily available from many 
user groups. 

Every program is evaluated by at 
least two educators before it is added to a 
dissemination disk and made available 
for copying. Programs arc edited for spell- 
ing errors, inaccurate or incomplete in- 
structions, errors in factual content, pro- 
gramming errors and other problems. 



Our goal is to distribute only those pro- 
grams which have some educational value 
and which are a,s free of errors as possible 
in both content and programming. The 
actual quality of the programs in the eol- 
lection varies greatly, as does program 
length. Each disk contains from 12 to 30 
programs, all for one system, but includes 
various subjects and grade levels. 

Donors complete a Preliminary In- 
ventory Form that identifies and des- 
scribes the program, and a Release Form 
that gives CUP! permission to distribute 
the program. The donor's form also asks 
whether the program is original, a modi- 
fication of another program, or an adap- 
tation from a magazine or book listing,* 

The contributed program is put onto 
a receiving disk where it will remain in its 
original contributed form as a permanent 
record. Filled receiving disks are placed 
into our permanent archives after all of 
the programs on that disk have been pro- 
cessed One donated program at a time is 
loaded from the receiving disk onto a 
working disk and all subsequent versions 
of that program will be stored on the 
same disk until the program is finished 
and ready to be transferred onto the dis- 
semination disk. We have approximately 
40 woiking disks, each with only one pro- 
gram, being evaluated at any given time. 
At least two educators will review the 
program on the working disk and make 
needed revisions. When they agree that it 
is finished the program will be removed 
from the working disk and added to the 
current dissemination disk in the SOFT- 
SWAP. Another program from the re- 
ceiving disk is then loaded onto that 
working disk and fhe evaluation process is 
repeated. All finished programs on the 
dissemination disks are available for copy- 
ing by visitors. 

Two forms are used in the evaluation 
of program: Checklist for Microcomputer 
Program Revision and Programmer Revi- 
sion Sheet, These forms list common er- 
rors and problem areas which evaluators 
are to check. Detailed notes describing 
needed revisions are made on the Pro- 
grammer sheet and the final reviewer 
checks to see that all requested correc- 
tions have been made. The finished pro- 
gram is described on a final Inventory 
Form - SOFTSWAP and then added to 
the SOFTSWAP catalog. Newly devel- 
oped programs are also listed periodically 
in the CUE Newsletter. 

These evaluation forms are not de- 
signed to produce critical reviews for pub- 
lication but to provide a guide to be fol- 
lowed in polishing the programs for class- 
room use. Programs are checked for ac- 
curacy of content, clear instructions, 
trapping of input errors, etc. Certain 
problems which frequently appear in con- 



♦Samples of the forms used for the SOFTSWAP 
are available from the authors at the San Mateo 
County Offiee of Fducation, 333 Main St., 
Redwood City, CA 94063. 
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Not Accepted 



Needs Major Revision Needs Minor Editing Finished; add to 

collection 

CHECKLIST FOR MICROCOMPUTER PROGRAM REVISION 



Name of program: 



Name of cassette or disk (if different from the program 
Reviewer's Name: 
Position : 



Student (age ) 

Teacher 



Administrator 
Other: 



Date : 



OKAY 



NEEDS 
WORK 



Make specific notations on yellow 
Write in comments as necessary. Programmer Revision Sheet. 



INSTRUCTIONS 

1. Are they given or are you told how to get them? 

2. Can they be bypassed if desired? Or are they short enough to be shown 
automatically? 

3. Can user return to instructions during program? 

4. Worded for the program's user group (age, class, etc.)? 

5. Complete and clear? 



B. INPUT 

1. Is there an indicator to show where input will appear? 

2. Do you get hints when you make input errors? 

3. Does program respond to illegal inputs? 

a. Are error messages easy to understand? 

4. Can you "fall out" of the program if you give bad input (e.g., just 
pressing RETURN) ? 

5. Does the input system allow for mistakes to be corrected by user? 



C. OVERALL IMPRESSION 

1. Is the format neat (no words running over the end of the line, lines 
double spaced for younger users, screen not crowded, etc.)? 

2. When comments, instructions, etc. appear on the screen for a limited time, 
is that time long enough to be read and understood? 

3. Does the program clear the screen at the start? 

4. Does the package make use of motivational devices? 

a. Timing 

b. Scoring 

c. Graphics 

d. Effective personalization (informal, conversational, addressing user by 
name, etc.) 

e . Random reinforcement 

f . Free of demeaning remarks 

5. Does the program have difficulty levels appropriate for its purpose? 

6. When appropriate, is correct answer given after a reasonable time or a given 
number of attempts? 

7. Does the program allow starting over easily? 



Checklist For Microcomputer Program Revision - p. 2 



OKAY 


NEEDS 
WORK 








C. Overall Impression, cont'd 








8. 


Is there a standardized instruction to continue (e. 
at bottom of the page? 


g. , hit spacebar) 






9 . 


Is program accurate? 

a. Spelling, punctuation, grammar, etc.? 

b. Is content accurate? 








10. 


Is it easy to end the program? Is it done neatly ( 


screen clean)? 



e circle the letter that most nearly indicates your opinion of this p 



SUMMARY EVALUATION 

A. Level of interest: 

interesting A B C D F uninteresting 

B. Ease of use (consider student, 
teacher, and setting up for next user) : 

easy to use A B C D F awkward 

C. Educational content and/or value: 

much A B C D F little 

D. Use of graphics: 

excellent use A B C D F no use 

E. Use of computer delivery: 

effective, can't be cone A B C D F there are better ways to 
as well by any other means achieve this objective 

OVERALL VALUE: every school should A B C D F not worth the effort 

have this program to load it 



Identify the strengths and weaknesses of this package. Please provide a paragraph summary 
of your reactions to the program. Thank you for being part of our evaluation tec 



2am . 
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COMPUTING IDEAS FOR EDUCATION 

CONDUIT Hir.'c tor J,vnr; W 

100 Ltnclijuist OnhT Assu<;i,»h« Dinner H;»okl 

The University ol luvwi 

POBox 388 WHAT IS CONDUIT? 

Iowa City. IA f)?2«M 
(319)353^789 

CONDUIT is a nonprofit organization that reviews, tests, 
packages, and distributes educational computer software. Begun 
in 1971 with support from the National Science Foundation (NSF) , 
CONDUIT is currently affiliated with The University of Iowa and 
funded in part by NSF and the Fund for the Improvement of 
Postsecondary Education. 

Above all, CONDUIT seeks to distribute quality materials. 
Our authors are typically college or university faculty who have 
created materials based on their own teaching experiences and who 
have used those materials with their students. All our packages 
are P ee r reviewed by leading educators for conceptual validity 
and instructional usefulness. They are also technically reviewed 
for program accuracy and transferability to a wide range of 
computer systems. 

The CONDUIT library currently includes 148 packages in 
biology, chemistry, economics, education, English, geography, 
languages, management science, mathematics, music, physics, 
sociology, political science, psychology, and statistics. These 
materials offer a variety of approaches tp instructional 
computing. Several packages are computer simulations of 
situations that are impossible or too expensive to reproduce in a 
laboratory. Other packages help teach quantitative techniques in 
such courses e.s linear algebra, calculus, operations management, 
and quantum mechanics. Some of our materials provide 
problem-solving tools or data for student research. We also 
distribute packages which tutor students in key concepts or 
provide drill and practice in basic skills. 

A typical CONDUIT package consists of a microcomputer 
program (for Apple, TRS-80, PET, or Atari) or a program written 
for standard mainframe computers (in Basic or Fortran); a student 
manual telling of objectives and methods of use; an instructor's 
guide illustrating course use; and notes describing installation 
of the materials on the computer. Although our packages are 
intended primarily for higher education, a number of high schools 
have also begun using some CONDUIT programs. 

In addition to distributing courseware, CONDUIT is 
continually studying ways to make instructional computing more 
effective. Our present activities include research and 
development of authoring aids, programming standards, transfer 
guidelines, and evaluation tools. In conjunction with this 
research, we publish a biannual magazine, Pipeline , featuring 
articles on new uses of computers in education. We also publish 
two reports on CONDUIT standards: the CONDUIT Autho r ' s Guide 
describing hov to design, develop, and package instructional 
software materials and the CONDUIT Bas ic Guide describing how to 
write and transfer programs in various Basic dialects. 
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® comduit 

CONDUIT Package Evaluation Form p.o. Box 388, iowa City, iowa 52244 

For 319-353-5789 

Microcomputer-Based Instructional Materials 

Package Title: 

Reviewer : 

Address: 

Area Code + Phone number 



I. Summary Assessments 

1) Do you recommend the use' of this package? 

s tro ng ly r ecomme nd 

recommend 

recommend subject to improvements (stated in section V) 

do not recommend 

Briefly explain your recommendation, idenci. 4 Sne 
strengths and weaknesses of the package. 

2) How central is the subject matter of this packaqe in vou~ 
field? J 

critical, absolutely essential 

important to include 

optional, appropriate but not essential 

trivial, not important 

Comment : 

3) Is it reasonable to use the 

yes 

no 

not sure 

Comment: 



computer with this package? 
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II * Description : 

1) Describe the topics or concepts presented in this package 
in a few words. (For example: The law of demand in 
microeconomic theory at the elementary level.) 



2) Suggest, the title of a course or courses for which this 
material is appropriate. 



3) Check the appropriate instructional level for using this 
package: (Check more than one, if appropriate) 

high school 

undergradua te (lower level) 

undergraduate (upper level) 

graduate 



other (Specify; 



4) What prerequisite skills are needed by students before 
they use the materials? 



5) State the instructional objectives of this package. 
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Ill - Course Use : 

1) Have you used the package with students? 
Yes No 



This section should be completed only if you have used 
the package with students. 

2) For each course in which you used this package, please 
state: 



Course title 

Level : 

Enrollment: 



Course title: 
Level : 



Enrollment : 



Course title; 
Level : 



Enrollment : 



3) Describe your students' reactions to these materials 



4) Did your students attain the learning objectives you 
specified above in section II, question 5? 



Yes No 



If not, can you determine why they didn't? 



5) State any suggestions you have for succesfully using 
the package with students. 
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IV . Evaluation of Programs ; 

Please rate the programs on each of the following characteristics, 




_ Substantiv e Content 

1. Definition of key concepts 

2. Discussion of underlying assumptions 

3. Validity of principles, theories 

4. Discussion of relevant literature 

5. Overall quality of substantive content 

B. Documentation/Textual Materials 

1. Clarity of information in textual 
materials 

2. Completeness of instructor guide 

3. Adequacy of instructions for operating 
programs 

4. Overall quality of documentation 

C. Support of the Teaching Process 

1. Ease of integration with course procedure 

2. Potential for improving instructor's 
ability to communicate principles and 
theories 

3. Potential for improving instructor's 
ability to communicate methods and 
techniques 

4. Potential for teaching how to interpret 

5. Overall instructional quality 
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D« Stimulation of Student Interest 

1. Potential for capturing student interest 

2. Challenge to student creativity 

3. Student choice in patterns of use 

4. Appropriateness for student-initiated wor 

5. Overall contributipn to student motivatioi 

E. Use of Graphics 

1. Motivational value 

2. Direct instructional value 

3. Aesthetic quality of graphics 

F. Use of Color 

1. Motivational value 

2. Direct instructional value 

3. Aesthetic quality of color 

4. Avoidance of distractive use of color 

G. Use of Audio 

1. Motivational value 

2. Direct instructional value 

3. Aesthetic quality of audio 

4. Avoidance of distractive use of audio 
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1. Motivational value 

2. Di rect instructional value 

3. Aesthetic quality of animation 

4. Avoidance of distractive use of animation 

I. Screen Layout 

1. General ease of reading layouts 

2. Attractiveness of layouts 

3. Avoidance of overcrowding 

4. Clarity of presentation 

5. Provision for student control of 
screen transit ions 

J. Student Interface 

1. Clarity of prompts 

2. Availability of help within program 

3. Handling of errors in student input 

4 . Provisions for student to edi t own input 

5. Acceptance of abbreviated input where 
appropriate 

6. Reporting of performance to student 
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Improvement 

For any characteristic you rated "fair 11 or "poor" in section 
IV, please recommend improvements to correct the problem. 



VI. Summary : 

Please provide a written summary of your general assessment 
of the package. Your summary should elaborate your evalua- 
tion of the substantive aspects of the materials. 
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National Council of Teachers of Mathematics; 

Guidelines for Evaluating Computerized Instructional Materials 



sect™ 3 

Gettang Hardnosed about Software: 
Guidelines for Software Review 



Getting the Message Out of the Medium 

Once basic documentation and a program for review have been obtained, the important process of 
evaluation can begin. The computer program is evaluated first, followed by a review of the related docu- 
mentation and ancillary materials. The evaluation serves only to provide information on which a decision 
can be reached; it does not use weighted scales or numerical totals to make the decision for you. 

The evaluation model presented in this section is both comprehensive and detailed. It should be 
tailored to meet the specific needs of a given user. In doing such tailoring, however, the user should be 
careful to ensure that the resulting version of the model considers each of six important steps of the 
review process. 

Step 1. Load the program on your system. Run the program briefly to become familiar with the pro- 
gram's "flow." 

, Step 2. Execute the program as a successful student would, avoiding intentional or careless errors. 
Extend the program when possible by interacting "creatively" as a good student would in testing the 
cleverness of the programmer. 

Step 3. Execute the program as an unsuccessful student would. Respond incorrectly to test how the 
program handles student errors. If an erroneous response to a task results in the repetition of that task, 
make repeated incorrect responses. Be sure to repeat the same response and also try giving different 
incorrect responses. Finally, make other kinds of errors such as typing mistakes, incorrect form of input 
(e.g., "one" for "1"), content errors, and errors in following the directions. 

Step 4. Use the checklist for the evaluation of software on pages 17-18, adapting it as necessary 
to fit your own needs. Throughout, your responses should be based only on your observations and 
experiences with the program. Once completed, your responses can be compared with the vendor's 
claims. 

Step 5. Complete the checklist for the evaluation of documentation and ancillary materials. Again, the 
extent of the review will depend on your needs and the impressions remaining from your review of the 
actual software. 

Step 6. Make a decision. 

The design of the sample Software Evaluation Checklist calls for items on one side of a sheet of paper 
and abbreviated explanations of item vocabulary or purpose on the other side of the sheet. (Detailed 
discussions of the terms used on this sample are presented in the next subsection of these Guidelines.) 
This one-sheet design is judged highly desirable insofar as it simplifies paper shuffling. (Of course, the 
items and their explanatory notes could be placed on your computer. This not only saves trees and 
storage space but also illustrates that you recognize another obvious advantage of computers in edu- 
cation.) 

Although the sample checklist here purposely includes an excess of items, in practice a shorter form 
may do the job. Moreover, experienced evaluators would require few, if any, of the explanatory notes 
given on the back of the sheet. 

Words to Evaluate By 

The use of our checklist, or any other checklist, by more than one person requires an agreement re- 
garding the meaning of terms used to classify or evaluate programs. We must extend the minilexicon 
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we started in section 2. We'll do this by proceeding item by item through the sample checklist, stopping 
to define and discuss concepts as the need arises. 

1. Instructional Range 

For any given program, the appropriate grade level and ability level must be judged according to 
the particular instructional setting. (A program that is informational at first grade might be remedial at 
third.) Key factors in determining grade. or ability level are the concepts in the program. Other important 
actors are reading level, prerequisite skills, degree of student control, and intended instructional use 
It is possible for a program to be flexible enough to be used across a wide range of grade levels and 
ability levels. 

2. Instructional Grouping 

The interaction between the learners and a program can be either direct or through an intermediary 
(the teacher). The first two groupings described below invoive direct interaction; the third uses an inter- 
mediary. 

Individual. Some programs are designed specifically for use by a single person. Programs of this 
nature may include a summary of individual performance; others may allow a specific level of difficulty— 
a specific set of spelling words, for example. Programs for individuals may have provisions for easier 
or more difficult situations, depending on the individual student's response. 

Small group. Some programs are specifically designed for use with two, three, or four students. One 
such example might be an arithmetic fact game involving a car race in which each student can move 
forward on a correct answer. Other programs for small groups might be business simulations in which 
each small group is a business. 

Large group. Large-group use of programs may vary in intent. A program may be used with a large 
group solely for the purpose of introducing the program for later use by individuals or small groups. 
Some programs are designed for use by a teacher in graphically demonstrating a concept. (Trigono- 
metric functions provide one example; fraction concepts such as equivalence are another.) 

3. Execution Time 

The time required for the use of a program will vary considerably. The load time of the program de- 
pends on the complexity of the program and on whether it is being loaded from a cassette tape or a disk. 
(The disk is a much faster but more expensive storage medium.) The type of program is a factor (e.g., 
tutorial versus informational). The user will require. additional time if responses are consistently incor- 
rect. Another time factor is the degree of user control (i.e., how many options are available to the user.). 
An estimate given as a range is an appropriate response for this item. 

4. Program Uses 

The classification of programs by the instructional uses to which they are put is difficult Careful 
analysis notwithstanding, definitions seem doomed to fuzziness, and a given program may seem to fit 
in more than one category. Moreover, the use of a given program can often be determined by the class- 
room practitioner; a program written with one use in mind may be used in completely different ways by 
each of several persons. In fact, a given person may identify more than one use for a program, leading 
to the necessity to indicate primary and secondary uses. 

Despite the difficulties just noted, there is instructional value in attempting to identify the use or uses 
of a program. The following definitions are proffered as an aid to the classification of those program 
uses that appear on the checklist. Additional uses are described later in this section. 

Drill or practice: Programs that assume previous instruction in the concept, skill, or process to be 
addressed. They present a controlled sequence of exercises designed to drill the recall of certain pair- 
ings (e.g., to associate French verbs and their English counterparts or to recognize musical notes 
aurally) or to practice some algorithm (e.g., to add two four-digit whole numbers, to sort buttons on the 
basis of two or more criteria, or to assemble a telephone). 
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Tutorial: Programs through which the computer assumes total responsibility for instruction. Tutorial 
programs are characterized by a dialogue between the student and the computer in which the direction 
and level of the dialogue are shaped by student input. Ideally, such programs would be high fidelity 
simulations of the best teaching behavior associated with a given topic and would control the variables 
associated with the what, why, when, and who of the instructional episode. 

Simulation: Programs that attempt to represent key aspects of some environment within which the 
user will experience the necessity to make decisions and will be informed of the results of those deci- 
sions without experiencing the real consequences of possible misjudgments. The time required to 
develop and use simulations with high fidelity is justified in situations where actual experience is ruled 
out because of extreme expense, safety considerations, or the time required for the actual experience. 
Simulations include problem-solving tasks (e.g., negotiation of a bank loan, diagnosis of illnesses or 
or equipment failures, genetic experiments, testing theoretical models), procedural tasks (e.g., acid 
titration, blasting, the breeding of organisms), and performances (e.g., control of water pollution). 

Instructional gaming: Programs calling on the user to apply one or more specific skills or concepts 
in a game environment. The game dimension of the environment includes the conditions of competition 
with oneself or others, specific (if arbitrary) rules, the need to develop a winning strategy, and the intro- 
duction of random events to force the revision of strategies. This means that the primary objective of the 
exercise should be the development of general problem-solving skills. This objective is often accom- 
panied by the objective of exercising specific concepts or skills in a new context. Examples of games 
that have been computerized include chess, Master Mind, and nim. 

Problem solving: Programs serving either the student or the instructor; the problem to be investigated 
may be either within a given subject or within instruction in this subject. In practice, there are two levels 
of use— one that employs existing programs using known algorithms (e.g., the distance traveled by an 
object in free-fall during a given period of time) and one that requires the development of a program 
employing a user-defined algorithm (e.g., a program that finds the sum of distances from proposed 
locations of six warehouses to each of the establishments to be supplied by those warehouses). In fact, 
the development of the algorithm itself may be the object of the problem-solving activity. 

Informational: Programs designed to generate information (lists of prime numbers, decimal approxi- 
mations of rational numbers correct to any given number of decimal places, powers or roots of a given 
number, synonyms for any word from a given list, a list of formulas or rules, etc.). Although such pro- 
grams might be employed to do problem solving or to construct a program in some other mode, the 
face value of the program remains that of data generation. They offer very little interaction with the person 
for whom the data is generated. 

A blank at the end of item 4 of the checklist invites the user to specify other program uses. Another 
six possible program uses are suggested below. 

Demonstration: Programs that assume the intervention cf an informed manager (teacher) between the 
program and the learner. They are designed to permit the manager to pursue the questions and sugges- 
tions of students by varying the conditions associated with some concept being developed. (For ex- 
ample, it might permit the variation of weather conditions in investigating their effects on annual produc- 
tion of selected crops.) These programs are generally devoid of textual display relative to their purpose 
and operation. This lack of text, while precluding direct use of the program by the learner, permits effi- 
cient realization of the program's purpose by an informed user (teacher). External documentation is 
critical to demonstration programs. 

Instructional management: Programs designed to aid the user in the flexible and efficient management 
of such limited instructional resources as time, equipment, workspace, instructional personnel, and con- 
sumable materials. Such programs assume, at least in part, the tasks of making instructional diagnoses, 
giving course assignments, evaluating student progress, keeping student records, and facilitating com- 
munication among instructors and students. 

Instructional support: Programs designed to facilitate the improvement of instructional products (e.g., 
worksheets, laboratory activities, newsletters, reports to students or parents), resource location (e.g., 
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listings of publishers, suppliers, materials, or human resources), and product selection (e.g., textbook 
readability, software evaluation). 

Test construction and analysis: Programs to aid in criterion-referenced testing or tailored testing. 
These programs offer such services as the banking of objectives or test items, item analysis, record 
keeping, test generation, and test scoring. 

Programming utility: Program building blocks that may be used to construct instructional programs. 
Their availability allows instructors or programmers to make efficient use of their hardware and their 
programming time. Examples of such building blocks would include programs that search a student's 
input for spelling, alphabetize lists, search lists for synonyms, create or delete data files, recall and dis- 
play selected geometric figures, and translate typed input into spoken words. 

Whistles and bells: Programs designed to acquaint the user with those characteristics of a computer 
system that may have instructional application. Such programs range from those that carefully demon- 
strate and explain a feature like screen protection (allowing sustained display of a chart or diagram with- 
out the possibility of student erasure or write over) to those that mindlessly exercise coloi and sound 
for the purpose of selling hardware. 



5. User Orientation: Instructor's Point of View 

Flexibility. A program may permit the user or instructor to adjust it to accommodate a range of ability 
levels, several degrees of difficulty with respect to a given user, or the class of concepts involved. This 
can be accomplished through options presented by the program. For instance, a program dealing with 
the practice of whole number computation might offer a choice of the operation (addition, subtraction, 
multiplication, division) or the range of numbers involved (0-10, 0-20, 0-100). Another option might 
involve the difficulty of the problems presented in terms of the procedure required for a solution (i.e., 
borrowing versus no borrowing in the subtraction problems). Programs that do not offer such options 
to the student may have a provision for the execution of such adjustments by the teacher. 

Intervention or assistance. Although it is a desirable goal for most programs to operate independently 
of the classroom teacher, there are programs for which assistance is needed and even desirable. For 
instance, some drill programs are "unending." Intervention to stop the program is done by the teacher 
and involves a judgment of what constitutes sufficient program use by a given student. (Note: A rating 
of "low" on "freedom from need to intervene or assist" would mean that considerable teacher interven- 
tion or assistance is required.) 



6. User Orientation: Student's Point of View 

Quality of directions. Directions need to be concise and clear. They should be developed for the 
lowest level of use. Some directions can be simplified by the use of an example. 

Quality of response. Responses to student input should be low-key and understandable. Responses 
to incorrect input should be neutral (e.g., "Incorrect. Try again." versus "Wrong, dummy! How could 
you get this far and know so little?"). 

Screen formatting. Formatting refers to the physical layout of text and graphics presented on the 
screen. Examples of poor formatting include a full screen of text with single spacing, too many graphics, 
continually flashing displays, and text scrolling off the screen. Some good examples are centering of 
data, a few items on the screen with the user in control of when the next items appear, and reverse-field 
printing of key items (i.e., light print against a dark screen versus dark print against a light screen). 

Need tor external information. A program may require the user to have access to information other 
than that printed by it. Generally speaking, the more self-contained a program is, the better. However, 
certain programs intentionally require the use of external items. Examples of such programs include 
programs dealing with dictionary skills and the use of maps. 

Disruption by system error. System errors result in the involuntary termination of a program. They may 
be introduced by improper calculations, errors in the logic of the program, jr input that is conceptually 
correct but does not meet the form required by the program. Programs written to prevent all systems 
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errors— syntactic errors and improper input— are the most desirable. They can be developed by 
using special input routines. 

Simplicity of student input. A program should ensure that a user knows whe:i and in what form input 
is needed. Characters with special meanings should be avoided. Input locations on the screen should 
be standardized throughout the program. Typing requirements should be minimal. If a special type of 
input is required, an example should be given. 

7. Content 

Instructional focus and significance. The program's topic should be clearly defined. The instructional 
objectives of the program must be viewed as important by the instructor. Also, the program should 
represent a valid use of the computer's capabilities. (The program should not repeat the same set of 
questions or present problems in exactly one sequence.) 

Validity and compatibility. The concepts and terms used should be correct, clear, and precise. The 
rate of presentation should be consistent with the levels for which the program is intended. The content, 
terminology, teaching style, and educational philosophy should be consistent with those generally en- 
countered by the student. 

8. Motivation and Instructional Style 

There are many options that may increase student motivation or develop a particular instructional 
style. Some of these options are a function of the hardware being used. Because thes3 features are 
optional, a column labeled "none," meaning the feature is not present, has been added at the left of the 
low-high continuum. 

Student involvement. The proper degree of involvement will vary with the type of program used. A 
dr practice program for one student is highly active, whereas an information program may require 
li-t ^dent input. Tutorial programs, being dialogues, should require student involvement. 

Student control. Several concepts may be involved here. Options available to the student at the be- 
ginning of the program may include different kinds of content, levels of difficulty, or the- type of response 
the student may wish to use. Additional examples of student control include the selection of the number 
of problems to be done and the provision of help (at the student's request) at any point in the program. 

Game format. The use of a game format often generates favorable responses from students. Some 
examples are mathematics drill as a drag race or in a tic-tac-toe or rntration format. Caution must 
be exercised so that the game format does not overshadow the instrui on. Note that a game format does 
not imply that the program use is instructional gaming; a drill program may use a game format. A focus 
on problem-solving strategies is required for a program to be classified as having an instructional gam- 
ing use. 

Graphics, animation, color. The functional use of one or more of these features can enhance instruc- 
tion; their improper or irrelevant application can interfere or cause misconceptions. The uses of graphics, 
animation, or color must be examined carefully to ensure that they support concept development. One 
might begin that examination by asking what would be lost if the feature were deleted. 

Voice and nonvoice audio. The use of voice input and output has considerable potential. Be aware 
that the quality and use of voice is dependent on the hardware. The use of nonvoice audio provides an 
additional means for attracting interest but may be distracting in the classroom. 

Light pen. Some programs require the use of a light pen. This can be a good feature if a light pen 
is available, but a program employing a light pen will be useless without it. 

Ancillary materials. Materials external to the program may be used. Some may be provided, but their 
use may not be required (e.g., pretests or posttests, worksheets, etc.). Others may be necessary but ate 
locally available (e.g., dictior aries or local road maps). Care 'should be taken to note any materials 
developed especially for the program that must be supplied by the program's author or publisher. 

As in item 4, a blank is left at the end of item 8. This is necessary because the list of special features 
grows daily; its length is a function both of the subject matter and the computer system involved. The 
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listed features are in relatively common use; other examples include touch panels (the user can execute 
an instructional task simply by touching the screen with a finger), special keyboards for the physically 
handicapped, electronic devices that monitor a physical phenomenon (such as water temperature) and 
use it in the program's execution, and mechanical devices (robots, if you will) that interact with the 
program. 

9. Social Characteristics 

Competition and cooperation. Some programs may be written for use by a single student, several 
students, or the whole class. Competition may be between two or more students or between a student 
and the computer; it is generally motivating. The need for cooperation among students, also desirable, 
might be found in a business simulation that requires several students to operate the business. 

Humanizing the computer. Programs may give the computer human characteristics. Some people find 
this desirable, whereas others wish to emphasize that the computer is a machine. Asking for the stu- 
dent's name may or may not be necessary or desirable. Older students generally find it boring, distract- 
ing, or phony. 

Moral issues. Items or techniques that at first create interest may be questionable from other points 
of view. Capital punishment as exhibited by most hangman games is one example; war names are an- 
other. 

Summary of student performance. A performance summary is a feature inherent to games that result 
in a winner. Performance on a simulation (e.g., the use of a credit card) also calls for a summary evalua- 
tion (e.g.. the credit card was cancelled because of the inability to meet payments regularly). Drill pro- 
grams often conclude with a summary regarding the speed or accuracy of the student's performance. 
Depending on the manner in which these summaries are presented, they may have either a positive or 
a negative effect on the learner. 

Notes on the Design or Redesign of the Checklist 

With the exception of the few descriptive requests at the top of the form and in items 1 through 4, 
the use of the checklist requires only making checks, circling words, or bracketing ranges of dots on 
judgment scales. The design of the scales in items 5 through 9 was with an eye to consistency; it also 
consistently expresses the summary prejudices of the writers. For each scale, a rating toward the left- 
most extreme (identified by such terms as "low," "passive," and "poor") would generally be, for the 
writers, a negative evaluation of the software. (This technique isn't foolproof. A rating of "low" on "free- 
dom from need for external information" might be desirable for a drill program but is probably neither 
desirable nor possible with a simulation.) That is, the writers felt that a program should rank high in 
flexibility, should call for active student involvement, should make good use (if any) of color, and should 
employ competition in a positive way only. This left-is-negative and right-is-positive arrangement of the 
scales allows one to get the general sense of an evaluation by scanning the completed checklist. When 
items are added or altered on the checklist, it would be a good idea to keep this design intact. Thus, 
should you feel that program control should be the purview of the teacher or that judgments should be 
within the program itself, you might reword the second part of item 8 to read as follows: 

low high 

freedom from student control 
as rewritten, the word low would continue to be associated with an undesirable characteristic. 
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SOFTWARE EVALUATION CHECKLIST 



PROGRAM NAME: 
SUBJECT AREA: . 



SOURCE: 



REVIEWER'S NAME: 



COST; 
DATE: 



i, INSTRUCTIONAL RANGE 



. grade level(s) 
. ability level(s) 



2. 1 TRUCTIONAL GROUPING FOR PROGRAM USE 



. small group (size: ) 

, large group (size: ) 

3, EXECUTION TIME 

minutes (estimated) lor average use 

4. PROGRAM USE(S) 

drill or practice 

_ tutorial 

simulation 

instructional gaming 

. problem solving 

informational 

other ( 
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5 USER ORIENTATION: INSTRUCTOR'S POINT OF VIEW 
low high 

flexibility 

freedom from need to intervene or assist 
6. USER ORIENTATION: STUDENT'S POINT OF VIEW 
low . high 

quality of directions (clarity) 
1 quality of output (content and tone) 
' quality cl screen formatting 

freedom from need for external information 
freedom from disruption by system errors 
! 3 simplicity of user input 



7. CONTENT 



high 



:> '■ instructional focus 

instructional significance 
soundness or validity 

compatibility with other materials used 
8. MOTIVATION AND INSTRUCTIONAL STYLE 



pass/ve 



low 



none poor 



acf/Ve 



high 



good 



type of student involvemeni 

degree ol student control 

use ol game format 
use of still graphics 
use ol animation 
use ol color 

use ol voice input and output 
use ol nonvoice audio 
use ol light pen 
use ol ancillary materials 
use ol 



9. SOCIAL \RACTERISTICS 



pre 1 



nof present and 
presen/ positive 



competition 
cooperation 

humanizing ol computer 
moral issues or value judgments 
summary of student pertorrnan^ 



Jl^nT^T^^ l6VelS ' 0r 3 Par " CUlar pf ° 9ram are pr,man| y de,ermined fa y lhe concepis involved. Other im- 

™? S£J,2f ?" 9 T ' r Q T [e SkillS ' d6gree ° f S,Udent con,ro, • and intended >nslructional use. I, ,s possible for a 
program to be flexible enough to be used across a wide range of grade levels and ability levels 

De ?sons 0 TaTmI a 9f m ar , e at d o S ' 9ne H ** US " by have been 0r can be modl,ied for P arllcl P a1 '™ by two or three 

usTd n mLV h?n ^ mula " 0ns ^ernonstranons 0 ).en pose opportunities for large-group interaction. A given program may be 
used in more than one grouping, depending on the instructor. y 

appVpria^ the °' 3 pr ° 9ram Wi " V3fy conSlderabl V- loading time for cassettes. A lime range .s the 

falL'S^hS^^i S 6 Cate9 ° nZed aCC ° rding ,0 ,he ' r uses Sorne programs ma * have more ,han one use - thus 
railing into more than one of the following categories: 

Drill or practice: Assumes that the concept or skill has been taught previously. 

Tutorial: Directs the full cycle of the instructional process; a dialogue between the student and the computer 

Simulation: Models selected, alterable aspects of an environment. 

Instructional gaming: Involves random events and the pursuit of a winning strategy. 

Problem solving: Uses general algorithms common to one or more problems. 

Informational: Generates information (data). 

5. These are factors relevant to the actual use of the program from the point of v.ew of an instructor 

or TouceX A Pr ° 9ram a "° W ° f ' he inS ' rUC,0r ,0 adjuS ' lhe program ,0 d,fferent abil "y levels ' de ^ es of difficulty, 

Intervention or assistance: A rating of "low" means considerable teacher intervention or assistance is required 

6. These are factors relevant to the actual use of the program from the point of view of a student 

u^.^^^^^ readab,e ' UndSr ^ «"*» **« ^ "a. scroll of, the screen unti, 
sh^e^^ ~' « « -Ponse to student input, the output 

3 Pr ° 9ram ^ Sh0U ' d n0t be d ' S,raC,mg ° r du,,ered Labels and s ^bo.s should be 

JTZfl nl0matm: fK Pf0gram m3y require ,he user ,0 have access ,0 information other than that provided within it This 
S^^sTd «S COnten, kn ° Wled9e ° r kn ° Wled9e °' C ° nVen,IOnS US6d by * he Pr ° gram deS ' gner « ^ "5b 
System errors: System errors result in the involuntary terminal n of the program. 

Input: A program should ensure that a user knows when and in what form input is needed. It should avoid usinn characters 
with special meanings, restrict input locations to particular screen areas, and require minimal typing 

7. These are matters relevant to the subject-matter content of the program. 

Focus: The program topic should be clearly defined and of a scope that permits thorough treatment 
JiQnif'cance: The instructional objectives of the program must be viewed as important by the instructor Also the program 
should represent a valid use of the computer's capabilities while improving the instructional process 

,e rTo^ be «"* a " d P-ise. Other important factors are 

* edUCa " 0nal Ph "° S0Phy °' Pr ° 9ram Sh0U ' d ^ COnS ' S,ent 
rum n C nTn P ^ e ! l l I IOn :K COOpera . , ?• and values are concerns ,hal ma y be a function of the way a program expresses them (War 

Sa 1 'b^ T a " S H mPle iSSUeS ' ) AIS °' ,hG " human ' 2,ng " ° f ,he W serve'for mo , a ^n o 
reduce anxiety, but it also may become tedious, misleading, and counterproductive. 

, J,^™ U T ar , y ° f , S,Ude ?! per,ormance can be d'Chotomous (win or lose), statistical (time expended or percent of items cor- 
rect), or subiective (as in the evaluation of a simulation). It may be for student, teacher, or both. 
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SCHOLASTIC SOFTWARE EVALUATION FORM 



The following "Scholastic Software Evaluation Form" is an exerpt from 
MICROCOMPUTERS IN EDUCATION: A SCHOLASTIC IN-SERVICE TRAINING PROGRAM by 
James Poirot and Karen Billings, published by Scholastic Book Services in 1982. 
The package contains a set of materials designed for educators to conduct 
their own computing workshops: a notebook for the planner/presenter, a set of 
participant handbooks, and slides for use by the presenter. The notebook 
contains objectives, lecture notes, workshop management suggestions,, forms, 
and activities for the following topics: Computers in Education; Software 
Evaluation; Hardware Evaluation; Programming in BASIC; Computer Literacy; Future 
of Technology in Education; and Planning for the Future. A section on resources 
lists books; periodicals; films; software publishers, software catalogs, and 
sources of software reviews; hardware manufacturers; and organizations and 
associations. In addition, purchase of the package includes ten 1-year sub- 
scriptions to ELECTRONIC LEARNING magazine. For more information, write 
Scholastic, Inc., 730 Broadway, New York, New York 10003. 
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SCHOLASTIC SOFTWARE EVALUATION FORM * 

I. IDENTIFICATION 

A. Program Name: 

Single Program Series 

B. Distributor: 

Name: 

Address: 



Telephone: 

C. Microcomputer 
Type: 



Memory Required: 16K 32K 48K 64K 

Special Language Required: 



Storage Medium: Tape Cassette 5" Diskette Module 

Other: 

Equipment Requirements: One Disk Drive Two Disk Drives 

Color Printer Voice/Sound Instrument 

Game Paddles Other: 



II. INSTRUCTION 

A. Grade Level 

K123456789 10 11 12 

B. Software Types 

— Drill and Practice Tool 

— Tutorial Diagnostic/Prescriptive 

Simulation Management 

Other: 



C. Curriculum Areas 

Language Arts __ Science 

Social Studies Career Education 

The Arts Foreign Languages 

_ Business Education Physical Education 

Mathematics Other: 

Reading 

D. Instructional Considerations 

Game Approach Classroom-Text Dependent 

Student Workbook Required Teacher Supervision Required 

*From MICROCOMPUTERS IN EDUCATION: A SCHOLASTIC IN-SERVICE TRAINING PROGRAM 

by James Poi rot and Karen Billings, Scholastic Book Services, 1982. 
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E. Special Characteristics 

— Timing Student Records 

Branching Material Modification 

Sound Capabilities 

F. Software Instruction Time 

Average Per Lesson Average Per Package 



III. EVALUATION 

For each criteria listed below, circle the appropriate Weight and Rating value. Multiply each Weight by 
each Rating to obtain a Product. Then add together the Weights and add together the Products to 
obtain the Total for each. 



A. Presentation Criteria 






Weight 




Rating 


Product 


Ease of Use 


1 2 


3 


4 


5 


12 3 4 


5 




Reliability 


1 2 


3 


4 


5 


12 3 4 


5 




Motivation 


1 2 


3 


4 


5 


12 3 4 


5 




Frame Display 


1 2 


3 


4 


5 


12 3 4 


5 




Documentation 


1 2 


3 


4 


5 


12 3 4 


5 






1 2 


3 


4 


5 


12 3 4 


5 






Total 












Total 


B. Content Criteria 














Weight 


Rating 


Product 


Accuracy 


1 2 


3 


4 


5 


12 3 4 


5 




Feedback 


1 2 


3 


4 


5 


12 3 4 


5 




Level of Difficulty 


1 2 


3 


4 


5 


12 3 4 


5 




Appropriateness for Computer Use 


1 2 


3 


4 


5 


12 3 4 


5 




Educational Standards 


1 2 


3 


4 


5 


12 3 4 


5 






1 2 


3 


4 


5 


12 3 4 


5 





Total Total 



To obtain an Average for Presentation Criteria, divide the Total of the Products by the total of the 
Weights. Repeat the process for Content Criteria. Add together the Average of Presentation Criteria and 
the Average of Content Criteria. Finally, divide the sum of the two Averages by 2 to obtain the Final 
Rating. 

Presentation Criteria: Total of Products/Total of Weights = Average 

/ 

Content Criteria: Total of Products/Total of Weights = Average 

/ 

Final Rating: ( + )/2 = 
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IN MANY WAYS, EVALUATING 

microcomputer software is like evaluating any other Kind of instructional material. You'll be 
looking for materials that are: • content-accurate; • appropriate for the grade level- 
Mjteresting; • free of bias; • and accompanied by thorough and well-organized teachers' 

But in evaluating microcomputer software, there are other aspects to consider that you 
won t encounter in other instructional materials. For example, since computers are inter- 
active you 11 want to see if the software makes use of the computer's uniquely interactive 
capabilities in ways that both hold students' attention and also instruct them 

You 11 also want to keep in mind that a program which excites inexperienced student- 
users today, may bore a more computer-literate class next year. "CAI has gotten a lot of 
oTt^ T*™ beC3USe kids ^ il "Cresting. " says Mark Falstein, editor of Courseware 
vn k 1, u ' "'? eW software review journal. "But as computers become part of our lives, 
it ll be old hat. In other words, Falstein says, programming should not rely on the novelty 
of computer delivery to captivate students, but rather on the interest level of the material 
and the manner in which it is presented. 

To help you get started in evaluating microcomputer software, we've provided a form on 
pages 48 and 49. The form represents a synthesis of the many evaluation forms available 
today (see note below), and of comments and concerns expressed most frequently by 
educators and instructional computer experts. The form can be helpful in two ways It can 
help you evaluate a piece of software that you are previewing for possible purchase. And 
once filled out, it can serve as a "hard-copy" record of your review, to be included in a 
school or district software library. 

Most of the form is self-explanatory. But, for the first-time user, here are some things to 
keep in mind as you use the form. 




Line-ui 

What Reviewers JL Look For 
When Evaluating Software 



Using the evaluation form 

P™ R a™m F T he form asks for some general background infor- 

X MA I ION about the program being reviewed. Be sure to mention any special 
hardware the program requires, such as a voice synthesizei or joysticks, and any special 
software, such as a language not standard on the microcomputer intended for use 

In Question 2, list any skills the program assumes knowledge of, e.g. does a math 
program require students to know how to graph? 

For Question 6, briefly discuss the appropriateness of the computer as a vehicle 
tor teaching the program's curriculum content and objectives. For example, you might 

(Continued) 
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Software Li neup 

(Continued) 

note: "Inappropriate; computer serves 
only as an electronic workbook," or 
"highly appropriate use of the computer 
for simulation." 

Part II of the form includes an Evalua- 
tion Checklist. Here, again, there are a 
number of questions to pay close attention 
to. For example. Question 3 under the Edu- 
cational Content section — "Is the difficulty 
level consistent for material, interest, and 
vocabulary?" — is the downfall of many oth- 
erwise commendable programs. "I saw a 
program whose thrust was reading prob- 
lems in math," says J. R. Pennington, 
principal, Decatur High School, Decatur, 
GA. "The program was beautifully done. 
But while the math was fifth-grade, the 
language was about first year of college!" 



A PROGRAM THAT 
EXCITES INEXPE RI-l 

ENCEI) STUDENT- 
USERS TODAY.. MAY ; 
IIORIv A COMPUTER 

LITERATE CLASS ' ' 
NEXT YEAR. ' • 



The Presentation section assesses the 
manner in which the program conveys the 
content. In answering Question 4 — "Do 
graphics, sound, and color, if used, enhance 
the presentation?"-^consider whether 
"Hash" is substituted for well-thought-out 
content. 

In the Interaction section, note question 
1— "Is feedback effective andappropriate?" 
Be wary of such feedback responses as 
"Wrong, dummy." "That's a totally unac- 
ceptable response," says Mark Falstein, 
Question 4 asks about student control of 
the pacing and sequencing of information. 
Good programs will allow students to con- 
trol the pace at which information is 
presented. 

In the Teacher Use section, pay special 
attention to the question about documenta- 
tion (Question 4). Good documentation will 
cover most problems that a teacher may 
encounter in using a program. It will also 
be written in "plain English." 

One final note. All review forms, ours 
included, will affect the way you look at a 
program. "The instrument tailors the re- 
sponse," says Stan Silverman, a New York 
teacher, "I once gave teachers different 
evaluation instruments and had them look 
at the same programs. I got all different 
responses from them. I try to teach teach- 



A District Software 
Evaluation Plan that Works 



OFTWARE EVALUATION IS 
. one problem that elementary school 
educators in Utah's Granite school 
district don't have to wrestle with. The 
district office in Salt Lake City takes care 
of it for them. 

B^h Ives, a soft-spoken man with seven 
ye* <perience in instructional computing, 
heaus the software evaluation program at 
Granite. The district is Utah's largest, in- 
cluding as it does the state's only large 
metropolitan area. There are 60 elementary 
schools in the district, staffed by 1,200 
teachers, and attended by more than 35,000 
students. More than 200 micros are at 
their disposal, and, according to Ives, 'That 
number is growing every week." 

One of Ives' tasks is keeping track of 
the vast number of instructional programs 
made available each year. He says he hears 
about such programs in various ways 
— primarily through ads in magazines, and 
sometimes through other teachers. 

If a program sounds promising, Ives 
tries to get a copy to preview. "We've had 
good luck getting preview copies, " he says. 
"I get them either from the companies, or 
from neighborhood computer stores." 

Most of these programs are then evalu- 
ated by a member of a reviewing commit- 
tee, which consists of two teachers from 
different grade levels and two school prin- 
cipals. Whenever possible, the program is 
reviewed by a teacher from the program's 
intended grade level. But other programs 
are reviewed by people outside the com- 
mittee who just want to help out. 

A standard three-part form is provided 
for the evaluation. Part One asks for basic 
information; Part Two asks more specific 
questions about the program's content and 
organization; and Part Three asks the 
teacher for a more subjective discussion 
of the program's strengths and weaknesses. 
"We devised parts One and Three here," 
Ives says; "Part Two I took from a journal 
for gifted students," 

Completed reviews are filed in a review 
library which consists of some 200 pro- 
grams. About four times a year; Ives meets 




with computer education facilitators — 
there is one at each elementary school — to 
keep them abreast of what's available and 
what's worthwhile. 

The computer facilitators in the Granite 
district serve as a liaison between the dis- 
trict office and individual schools, Teachers 
at the elementary schools who are looking 
for software first turn to their facilitators; 
then, when they narrow down the pro- 
grams they're interested in, they request 
an evaluation from the district office. 
The decision on whether or not to pur- 
chase the software is then made by the 
staff at the schools, rather than by the 
district office. 

Granite's software evaluation program 
is already a model of simplicity and effec- 
tiveness, but Ives still isn't satisfied. This • 
year, he's planning to put software evalua- 
tions on diskettes to be made available 
quarterly to schools with micros. Soon, he 
says, software evaluations may also be 
available on a data base being assembled 
at the University of Utah — on-line for 
schools with modems, and in hard-copy 
form for others. 

— R, Neumann 



ers to look at the programs, and not be 
misled by forms." 

Most computer-using educators agree, 
however, that an evaluation form, like the 
one on the next two pages, does have its 
place. It can provide you with some gen- 
eral guidelines for reviewing new programs; 
it can help you build your own set of cri- 
teria for judging software; and it can pro- 
vide you with a handy record of programs 
in your software library. 



Editor's Note: The followingeducationalorganizations. 
institutions, and magazines have developed forms for 
use in the review and evaluation of microcomputer 
software: (1) California Library Media Consortium for 
Classroom Evaluation of Microcomputer Courseware. 
San Mateo County Office of Education. Redwood City, 
CA; (2) The Computing Teacher. University of Oregon. 
Eugene. OR; (3) Curriculum Review. Palo Alto, CA; 
(4) ERIE Institute and Vicki L Blum. Stony Brook. 
NY; (5) Microcomputer Resource Center, Teachers 
College. Columbia University, NY; (6) MicroSIFT. 
Northwest Regional Educational Laboratory. Portland. 
OR; (7) The National Council of Teachers of Mathe- 
matics, Reston. VA, 
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Reviewer's Name:. 
Address/Phone: 



_ Date of Review:. 
-( ) 



Program Title. 



Package Title. 



_ Medium: 5" disk; 8 "disk; 

cartridge; tape 

_ Copyright Date (if any) . 



Microcomputer (brand, model, memory) 

Necessary Hardware _ Necessary Software. 

Producer 



Back-up Copy Policy 



_ Author(s)_ 
-Cost 




Program Overview and Description 



1. Subject area and specific topic. 

2. Prerequisite skills necessary. 



3. Appropriate grade level (circle) 123456789 10 11 12 college 

4. Type of program (check one or more) 

Simulation Testing 

Educational Game Classroom Management 

Drill and Practice 0ther (sp ecify) 

Tutorial 

Problem Solving 

Authoring System 

5. Appropriate group instructional size: individual. 



-Remediation 
.Enrichment 

-small group class 

o. Is this program an appropriate instructional use of the computer? 



7. Briefly list the program's objectives. Are they clearly stated in the program or 
in the documentation? Are they educationally valuable? Are they achieved? 



8. Briefly describe the program. Mention any special strengths or weakne 



sses. 
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Evaluation Checklist 

Please check Yes, No, or Not Applicable for each question below. To add information, or to 
clarify an answer, use "Comments" at the end of each section. 

Yes No N/A EDUCATIONAL CONTENT 

_ 1. Is the program content accurate? 

. 2. Is the program content appropriate for intended users? 

3. Is the difficulty level consistent for material, interest, and vocabulary? 

4. Is the program content free of racial, sexual, or political bias? 

Comments: __= 



Yes Wo N/A PRESENTATION 

1. Is the program free of technical problems? 

. 2. Are the instructions clear? 

3. Is the curriculum material logically presented and well organized? 

4. Do graphics, sound, and color, if used, enhance the 

instructional presentation? 
5. Is the frame display clear and easy to read? 

Comments: = — 



Yes No N/A INTERACTION 

1. Is the feedback effective and appropriate? 

2. Do cues and prompts help students to answer questions correctiy? 

3. Can students access the program "menu" for help or to change 

activities? 

4. Can students control the pace and sequence of the program? 

5. Are there safeguards against students "bombing" the program 

by erroneous inputs? 



Yes 



No 



N/A 



TEACHER USE 

1. Is record-keeping possible (within the program or through 
documentation worksheets)? 

2. Does teacher have to monitor student use? 

3. Can teacher modify the program? 

4. Ms the documentation clear and comprehensive? 



Comments: 



Parts 



Overall Evaluation 

CHECK ONE. 

Excellent program. Recommend without hesitation. 

Pretty good program. Consider purchase. 



.Fair. But might want to wait for something better. 
.Not useful. Do not recommend purchase. 
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Three samples of software reviews are presented 
on the following pages. The reader will note a 
number of contrasts among the reviews: the num- 
ber of categories of evaluation, the amount of de- 
scription, the formats for presenting evaluation 
summaries, and the relative emphasis on selected 
features of software differ considerably among 
these samples. Because the information in soft- 
ware reviews can differ from source to source, 
consumers of software evaluation data might do 
well to examine several different published reviews, 
in light of the information needed to inform local 
software decisions. In addition, it may be possible 
to find several different reviews for the same 
software. 

Each of the sample reviews is the product of an 
organized system for making evaluations of soft- 
ware available to educators on a regular basis. 
These systems share the following characteristics: 

© program description 

© category schemes that logically group program 
features 

® criteria for evaluation of software features 

© rating procedures and/or forms to summarize 
evaluative information 

• specific procedures for conduct ing evaluations 

• processes to train and maintain groups of 
professional software evaluators 

• systems to maintain files of and/or publish and 
distribute evaluations of selected educational 
software 
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The first example, MicroSIFT, represents a non- 
profit institutional approach involving an exten- 
sive network of organizations and individuals con- 
cerned with software evaluation. School Micro- 
ware Reviews represents a commercial organiza- 
tion that operates an evaluation system dependent 
on evaluative input from participating educators. 
Courseware Report Card, another commercial sys- 
tem, employs staff and consultants to develop 
software evaluations. 

All of the systems represented make their evalua- 
tions available to educators, MicroSIFT makes 
their software evaluations available through the 
network of cooperating institutions, and through a 
public access database called Resources in Com- 
puter Education (RICE). School Microware Re- 
views and Courseware Report Card publish collec- 
tions of reviews at regular intervals, which are 
available on a subscription basis. School Micro- 
ware Reviews also makes its evaluations available 
to software publishers and encourages dialogue 
between reviewers and publishers. 

Each of the sample evaluations on the following 
pages provides a glimpse of a specific software 
product. As such, they are a good starting point in 
the process of deciding what software to use in the 
classroom. But no single review can provide all the 
information necessary to decide if a particular 
product is the right one forits intended use. These 
evaluations can help educators identify potential 
selections that can be examined further at the local 
level by software review committees and indivi- 
dual teachers. Permission to reproduce the informa- 
tion in this section was granted by the publishers. 
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microISlFT COURSEWARE EVALUATION 



Arithmetic Racing 



PRODUCER: Math Software 
1233 Blackthorne Place 
Deerfield, IL 60013 

LOCAL DISTRIBUTORS: Contact producer for list 

EVALUATION COMPLETED: Fall 1981; Revised 
2/1/82 

VERSION: © 1980 

COST: Not sold individually; sold in packages of 5 
to 10 programs ranging from $100 to $250 

ABILITY LEVEL: Grades 4-11 

SUBJECT: Mathematics: speed and accuracy 

drill of arithmetic operations 
MEDIUM OF TRANSFER: 5" flexible disk 
REQUIRED HARDWARE: 32K Apple II or II Plus, 

one disk drive, monitor 
REQUIRED SOFTWARE: DOS 3.2 or 3.3, 

Applesoft 

INSTRUCTIONAL PURPOSE: Standard 

instruction, enrichment 
INSTRUCTIONAL TECHNIQUES: Game, problem 

solving, drill and practice 
DOCUMENTATION AVAILABLE: Suggested 

grade level, program operating instructions, 

demonstration 

INSTRUCTIONAL OBJECTIVES: To improve 
students r arithmetic skills in addition, 
multiplication, subtraction, and division; to 
provide an interesting interactive environment for 



remediation work in arithmetic facts; and to 
develop speed and accuracy in working basic 
arithmetic operations. 

INSTRUCTIONAL PREREQUISITES: The program 
assumes that students know basic arithmetic facts 
concerning the operations of addition, 
multiplication, subtraction and division. Students 
also need to understand the rules governing the 
operation of the computer game. 

CONTENT AND STRUCTURE: ARITHMETIC 
RACING is a game of timed arithmetic practice 
for students Grades 4-11. Players first select 
addition, subtraction, multiplication, or division 
and then specify the largest number they want the 
computer to give them. Players also select a 
speed level from 1-5. The computer then assigns 
a point value to each problem based on these 
selections. A 25 point bonus is added to the score 
for answering each of the ten questions correctly. 

ESTIMATED STUDENT TIME REQUIRED: Not 
available 

POTENTIAL USES: The program may be used in a 
classroom setting to provide drill and practice in 
basic arithmetic operations. 

MAJOR STRENGTHS: The game format offers an 
incentive for students needing drill and practice 
in arithmetic operations. 

MAJOR WEAKNESSES: None cited. 



EVALUATION SUMMARY 
SA A D SDN A 



SA A D SD NA 













Content is accurate. 












Content has educational value. 










e 


Content is free of stereorvpes. 




• 








Purpose of package is well defined. 




• 








Package achieves defined purpose. 




a 








Content presentation is clear and logical. 




o 








Difficulty level is appropriate to audience. 


• 










Graphics/sound/color are used appropriately. 




• 








Use of package is motivational. 




• 








Student creativity is effectively stimulated. 




• 








Feedback is effectively employed. 





e 








Learner controls rate and sequence. 




o 








Instruction integrates with prior learning. 




o 








Learning can be generalized. 




c 








User support materials are comprehensive. 




o 








User support materials are effective. 




e 








Information displays are effective. 




e 








Uv*> can operate easily and independently. 




e 








Tea_.icrs can employ package easily. 




• 








Computer capabilities are used appropriately. 




• 








Program is reliable in normal use. 



SA - Strongly Agree A -Agree D - Disagree SD - Strongly Disagree NA - Not Applicable 



Evaluators indicate they would use or recommend use of package with little or no change. 

fif!W| Northwest Regional Educational Laboratory 

300 SAV. Sixth Avenue ■ Portland, Oregon 97204 



(503) 248-6800 
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This evaluation is based on the evaluations of three or more reviewers 
who are representative of potential users of the courseware package. 

Permission to reproduce this document is hereby granted. 

7S 



four Name_ 
Address 



SCHOOL MICROWARE EVALUATION FORM 
0 rgani zation 



(COPY A3 NEEDED J 



Tel : 



Product Name u'nat Different supplier 

Subject:. 'J ind Grade 3 c srud^n — 

to Which Applicable j |fl n ■ r t ; a ^rp M Tt S - Rttrl f l j H £j C QLip Te h fi Q S j Q nima Required^ 

FUNCTIONAL DESCRIPTION'- Describe the program in terms of its goals and what it Joes 
as mucn detail as possible (use extra sheets as necessary). 



, r , r No. ot Progs . , 
.Price S lb.Ub Under This Name lU Segments 



to achieve them. tjivn 

1'he student is ffiven four words, three of a similar group of things and 



one which does not belong in the group. He types in the number of the 
word tnat is different fro m the others. Lach segment requires making 



increasingly difficult choices. Vocabulary words are grade 2 or lower (Cont. 

in^cTy'f schoof lP^s" ^ g" qram """"bate to the teacnxnc of topi cs, which should 55 taucn t Over) 

end'o^ne ror^nd cSl" theT^n i "ne^es t IZa ire? 0 " 3 ^ """ r l " Ch " CO "^ tS " " e 

I NSTRUCTIONS 



, nnnnn ln „ H Sn ^ er a number in the blank to indicate the extent to which the product fulfills thedes- 

• ; > L s r " :"-v' r '» - tK"H-.; "^'s: '^jk - - - — - 

ments at end or on extra sheets, giving item numbers. 



necessary m Com- 



DOCUMENTATION 
. g. , 



prog ram, 




i . 


9 


a 


2 


b 


2 


c 


1 


d 


u 


e 


1 


f 


0 




I 


h 


ij 


i 






in' / A 


j- 


J 


K . 


G 


1 . 


-2- 


2 . 



- List materials accompanying the 
teachers guide, student workbook. 



Suggested course/subject, grade levels 
Goals . 

Performance objectives. 

Suggested teaching strateg ( ies ) . 

Correlation with standard texts. 

Prerequisites for, use of program. 

Student exercises, teacher answers. 

Operating instructions. 

Listing and sampt« r'jns of orcgramfs). 

If a simulation, description of the 
model used. 

Suggested topics for p ~*_I->w-up 
discissions. 

Suggested references/activities for 
f o 1 low-up . 



3- If a workbook is included, the format 
and content are appropriate. 



2 5 

w/A_ e 
_ _2_ 7 

2_ 8 

2 9 



INSTRUCTIONS GIVEN TO USER By PROGRAM 

L. The instructions are adequate regarding: 
a. The instructional task to be performed. 



2 
1 



b. Details of how to interact with the 
program. 

U 2. User has the option of skipping 
instructions if already <nown. 

STUDENT-COMPUTER DIALOG 

2 1. Output is displayed screen oy screen 

(paged) rather than scrolled. 

2. If output is paged: 

2 a. User has control over continuing to the 
next page. 

2 b. Amount of information in each page is 
appropriate. 

2 c. The perceptual impact (amount of type 
— and lines} is suitable. 

2 3. Output is spaced and formatted so as to be 
easily readable. 



2 4. Language is we 11 suited to most students' 
read i ng ab i 1 i ty . 
Uses correct grammar , spelling , 
hyphenation and punctuation. 
Any grid or coordinate system used is 
consistent with common conventions. 
Students can respond with common symbols & 
ways of using them, e.g., right to left 
en try of sums . 

Accepts abbreviations for common 
responses . 

Provides for individual needs, e.g., 
opportunity to work with harder or easier 
ma terial . 

J 10. Dialog is personalized, i.e., makes 

appropriate use of student names. 
0 11. Uses devices to get & maintain interest, 

e.g., variation of computer responses, 

humor, pace change, surprise. 
12.rtaXes good use of any special features 
computer : 

a. Graphics H / £p . Color [\/^c. Sound 

2 13 .Reinforcing responses { indications of 
— — right, wrong, etc.) are appropriate . 

2 14. The number of wrong answers allowed i s 
reasonable. 

U/ A 15 -Responds appropriately if allowed number 

of wrong answers is exceeded. 

N / A 16 * Prov ides opportunity to get helo if 
LLLtX difficulty is encountered . 

2 17 .Minimizes bad entries via devices such as 

objective formats (multiple choice, etc). 

iJ/A 19 - Deals well with inappropriate entries, 

L£\ i.e., response to typing errors, etc., 
is intelligible and useful. 

2 19. Required entries are within students' 
capabilities (esp. typing , vocabulary ) . 

2 20. Reports student performance periodically 
and at end of session. 

MISCELLANEOUS CONCERNS ' 

If a simulation, the program gives a 
sufficiently accurate representation of 
the situation si mu la ted. 

2 2. The concepts and vocabuLary required to 
use the prog ram are reasonable. 

2 3. Operates properly and is free of bugs. 

2 4. Is woll structured and documented 

internally to facilitate any necessary 
debugging/modification. 



COMMENTS - Please use this space and additional sheets as necessary to provide any information which you 
believe would help someone wr.o is thinking about buying of the product being reviewed. In particular, 
indicate what you like best ar.-l least about the program. Also, list any changes which should be made. 

The stated goal of this program is to build reading comprehension and 
lo^ic skills. It would be more accurate to say that the program impro ves 
vocabulary and logic skills. Students from Uth to 8th grade have en- 

revised 9/82 ( Coilt . Over) 
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Functional Description ( Continued) 



in game one and gradually work up to grade 6 or lower in games 8-10. 

If a student gives a wrong answer the program requires him In try 
again until he gets the problem right. Every wrong answc 
counted in the score. 

One very nice feature in the program is that after the stui ^ m gives 
the right answer the program tells him why his answer is right 
(e.g. 1. red, 2. blue, 3. clear, 4. green 3 is correct because 
clear is not a color) . 

At the beginning of the segment the student has an opportunity 
to choose the number of the question v/ith which he wishes to 
start, allowing him to bypass questions on the lower segments. 
The user can also stop before the end of the segment and receive 
his score up to that point. 



Comments (Continued) 

joyed using this program and it provides good practice in 
classifying words. The only drawback is one of format. Every 
question is first marked off inside a box and there is a slight 
delay before the words appear in the box. Some students find the 
waiting somewhat irritating. A little more interaction in the 
way of personalizing the program and rewarding the user would be 
welcome. The student should be told that his score will appear 
at the end of the segment and that every wrong answer will 
count, not just one per problem. 



o 
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150 West Carob Street * 'Jompton, CA 90220 



Subject Area: Social Studies 
(trade Level: 8 and up 
Type of Program: Simulation 
System Requirements: 

• TRS-80, 48K, disk 

o Apple II, 4SK. disk (DOS ;U) 

* ATARI. 40K, disk; 'MK, tape 

Price: $^4.95 

Publisher: 

Creative Computing Software 
39 E. Hanover Avenue 
Morris Plains, NJ 07950 



Hav to the Chief 

Creative JoLiputinc Software 



5 : n-uiation >f ar: election campaign in which 
t'\e stutii---- . become a c and: 'late for president, 
'he stu :. : muse declare positions on real-life 
impaign *sues and use program-generated 
pinion polls tc determine the strategy neces- 
sary to win thr* election. 



(t's a rough business, be:ng a presidential candidate. / 
position on energy poV.cy. Somesharp! .porter points c, it th 
last week. Attempting to recoup, you lav out $150,GOO fror \ 
ally televised address on military preparedness. The pubi. 

These are two of the fit falls to be avuided in Ha:- to th° ' 
election campaign from Creative Computing Soft ware, 'i 
running against a program-generated opponent. 



a ru vs conference you're asked r-j clarify your 
tthep siUo i ir, inconsistent with chconeyou took 
ur carefully budgeted campaign fund for a nation- 
.■'3 response? "An actor ^ould have done better!" 

hirf, in jutstan Jiiif? simulation of a presidential 
ifjdent-play.-:r becomes a presidential candidate 



. .„ _ ... 0 rr „ Thepr;v,;: 3rri incorporates su:h real-life political concerns as 

public image, regional interests, campaign spending lirrvtb, :he advantage of incumbency, issues, and debates. 
r l he "candidate" is able to see the results of decisions rejected in nation^iopinion prlis and to plan strategy ac- 
cordingly 

Description of the Program 

The game can he played at any of four levels of complexity. Lev eh 2 and u P a.ssign the player to either the 
Democraticor Republican party; at level 1 there is no party assignment. Levels 3 and 4 add the factor of incum- 
ber . y; either the player or the programmed oppor.ent is the sitting president oris supported by the incumbent 
Thf higher levels also add the problems cf campaign funding and staying within spending limits. At each level, 
the player may choose fro rn among ten degrees of diffic .<ity. The higher the difficulty number, the harder it is to 
win: the player may be pitted against a popular incumbent or assigned by the program to the Democratic party 
in an era of Republican conservatism. 

The player must take positions on ten campaign issues (se<> Figure 1), each defined by a brief statement in the 
program manual. The player takes a position on each issue oy entering numbers ranging from -70 to +70. (It is 
perhaps indicative of the viewpoint of the program's creators that the negative n umbers indicate liberal posi- 
tions while positive numbers indicate conservative positions. An 
entry of zero represents a middle-of-the-road position.) At level 1 
the player enters positions on all ten issues at the start of the game. 
At higher levels the player initially declares positions on only three 
issues but will be asked to declare positions on other issues during 
the game. 

The program simulates a nine- week campaign, from September 1 
to Election Day, Tuesday, November 4. The program proceeds by 
increments of several days; each new date is displayed on the screen. 
In each time-frame, as they used to say in the Nixon White House, 
the player must select a particular campaign strategy to pursue (see 
Figure 2). These include televised appearances, regional ad cam- 



Performance: 


B + 


Ease of Use: 


A 


Error Handling: 


A 


Appropriateness: 


A 


Documentation: 


C 


Educational Value: 


A 



• 1982 Educational Insip- Not to be reproduced in any form without permission 
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WHICH ISSUE DO YOU CHOOSE NEXT 
\ FOR LABOR 

2 FOR DEFICI'i SPENDING 

3 FOR I1ILITAR. APPROPRIATIONS 
FOR INFLATION US UNEMPLOYMENT 



MIT! 



5 FOR STRATEGIC ARMS Li 

6 FOR WOMEN'S RIGHTS 

7 FOR NUCLEAR POLICY 
3 FOR MID-EAST POLICY 
3 FOR ENERGY POLICY 

10 FOR HEALTH SERUICES 

ISSUE NUMBER?! 



Figure 1. Player must cake positions on ten cam- 
paign issues. The issues arc defined by state- 
ments included in the program manual. 



9/3 



WHAT STRATEGY DO YOU WISH TO USE TODAY? 

1 FOR TU 

2 FOR ADS 

3 FOR TRrt'JEL 

4 FOR POSITION PAPHR 

5 FOR POLL 

6 FOR RAISING FUNDS 

STRATEGY NUMBER? 1 



F i s u re 2 . Th e s ix corn pa ign strut egi es available 
to the player. "Current date" is displayed at upper 
riyjit. 



MONDftV POLL 



REGDN 



EAST 

MIDWEST 

SOUTH 

PLAINS 

WEST 



NATIONAL 



YOUR 
STANDING 

49 
55 
46 
'56 
^ 3 



OPPONENT'S 
STANDING 

51 
44 
54 
44 
47 



49 



(PRESS ftHY KEY \Z CONTINUE) 



paigns, speaking tours, position papers, taking a 
national poll by issue and region, and (on level 4 only) 
fund raising. In addition, the program will randomly 
throw in such events as debates and news conferences 
in which the player must answer questions on the is- 
sues. In some cases the program gives immediate feed- 
back on the results of strategy: "A very poor showing"; 
" A QMite effective appearance." More specific results 
are shown in the straw poll (see Figure 3) displayed 
every "Monday," showing the candidates' standings in 
five regions of the country: EAST, MIDWEST, 
SOUTH, PLAINS, and WEST. (The manual includes a 
tabulation of electoral vote by state and region so that 
the player may plan regional strategies.) The effective 
learning experience of the game is in the player's ana- 
lyses of these results in order to plan subsequent strate- 
gy for the campaign. 

The following example illustrates how this works: A 
candidate identified as a relatively liberal Democrat 
makes a national TV appearance. The program asks 
him to select two issues on which to state his position 
(on the -7G/ + 70 continuum). The computer judges it "A 
quite effective appearance." The following Monday 
poll, however, shows an erosion of support in the con- 
servative SOUTH »nd PLAINS regions. Now the can- 
update chooses to take an in-depth poll that shows re- 
gion-by-i-e^on op ion on ^oh issue. Through this 
poll the candidate determines that hio positions on 
military spending and energy are more consistent with 
opinion in the SOUTH and PLAINS than his positions 
on other issues. He then may select making a speaking 
tour of those regions, or, if funds are short, taking out 
ads in those regions, clarifying his positions on those 
particular issues. Sure enough, the next Monday poll 
shows his standing in the SOUTH and PLAINS to be 
markedly improved. 

In the next time increment, the program declares a 
news conference. Our candidate is asked his opinion 
on Saudi oil. He responds with a " + 20," reflecting his 
relatively conservative energy policy. However, this 
response is seen as beinginco sistent with his strongly, 
pro-Israel Middle Ea.u stance. "WAFFLER!" the pro- 
gram declares, "YOUR POSITIONS ARE INCONSIS- 
TENT!" The next poll shows an erosion of support in 
the generally pro-Israel EAST. The candidate may 
then choose to travel to the EAST to reaffirm his sup- 
port of labor and of government- supported health care. 

The program continues in this manner until Election 
Day, when final results are computed and displayed 
and electoral vote broken down by region. A losing 
candidate is subjected to a funeral dirge, while a winner 
is treated to the strains of "Hail to the Chief and the 
display "CONGRATULATIONS, PRESIDENT- 
ELECT.' q 

O O 



Figure 3. The Monday poll. 
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Fvaluation 

Performance: The program runs smoothly. The student has as much time as necessary to read and ana- 

StT'rn m n eru ; rr s on thesereen until thestudent the ^^^1^ 

f\rth P , KF T t0 S1XtV mmUteS ° r '° nffer t0 run ' however, and the length of time it takes 

or the computer to analyze responses will f. ,t the patience of some students. A nine-week campaign may par 
ahel the real thing, but m this format, especially at the lower levels of complexity, it tends to be repetitfve 

Ease of Use: Limited only by a students reading ability or understanding of the concepts. All required re- 
sponses are in the form of one- and two-digit numbers. Menus and instructions make the meaning of each - 
sponse clear. It a student is confused, typing HELP will call up an explanation. 

JZ^RfIvtI WC [ eVieWed tht ' APPle V6rSi0n and f ° Und U im P— hie to crash the program short of 
rutting the RESET key. Any inappropriate response is met simply with a reminder to USE NUMBERS and 
with a repetition of the question. umor.no ana 

mtZ^ Pri ? tene u ^ r^rr thLS LS W , hat tom P uterb ^ instruction should be about. No textbook or 
f.lmstrip can possibly deal with An, an electoral politics in such a dynamic and interactive wav The pro- 
gram s creators have included an impressive array of situations and variables that could be incorporated into 
suchruuena.oniybymeansofacomputer. Sound is limited to a few brief musical phrases and a beep or two- it 
should not oe terribly distracting ,n a classroom. Apple and ATARI versions include a color display of he 
V h.te House w.th the title; the remainder of the program is entirely text. While this makes for humdrum video 
the older students, for whom the program is intended, should be able to take it. Memory-eating color graphics 
might have cut :nto the programs content. grapnics 

Documen'otion: The eight-page manual explains the basic rules of the game q uite adequately butsomeof 
the game sm.portant features have to be learn,d through trial and error. For a program that taL so !ong to 

TthT,:::" tnd rn nstruclions f cou d warn a piayer to be sure to wnte down the ™* h * r * ™^ ^£2™ 

on the ussues. and those given for the opponent. Any departure from these initial positions is regarded as 
waffhn,. Worksheets with space to record this information might have been included, as well as a few hints on 
campaign strategy as ,t affects the flow of the game. As it stands, a player can waste an hour or more of class 
time figuring out how the game works. 

Educational Value: The program's incorporation of many real-life situations and its balance between 
direct feedback t<, udent responses and :he presentation of data requiring interpretation make Hail to the 
Chuf at best simulation program w e ve ever seen. There are no aro.trary gains or losses, as in such highly 

o Te dtrZT* I" 0 "«°" : P ! ay er 1S ,f blet ° S6e ' dlreCtly and indireCt1 ^ the consequences of his 

or her decerns. The various levels of complexity allow the game to be played by students over a range of grade 

levels and abilities. Instructions both on-line and ,n the manual are clear and free of errors. This program > 

an effective and dynamic supplement to any unit on American government and the electoral system The only 

reservation we have is that it as too real a simulation: The public relations and media manipulation required to 

win are almost as Machiavellian as in an actual election campaign. Principles and ethics do not enter into this 

simulation at al image and cleverness are the only relevant value. The program's response to a player's bad 

TV showing ,« An actor could have done better!" Students will recognize that in real life one did 
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Books 

Burke. ILL. C \l Sourcebook. Englovvood Cliffs, NJ: Prentice-Hall, Inc., 1982. 

Includes a section on "CA( Cor \sew-re Review." 
Coburn. e. al ['radical Guide towpu^m in Education. Reading, JCfA: Addison-VVesley, 1982. 

Contains a discussion of tduational sof'twye selection. 
Criteria for Evaluating and Meeting Micro* mputcr Courseware. Baltimore, Maryland: State Depart- 

DouRlas, Shirley and (.ary Wights. A Guide to Instructional Mi. rn, .mputcr Software and .4 Guide to 
ofTdrcTiori%0 nna ' VlVania Depar ' ment ° f Educati0 ". reprinted by Connecticut Department 

!ys"e C ms Cti0n ^ im P lementi "g microcomputer-based instructional 

Edwards, Judith B. e! al Computer Applications in Instruction: A Teacher's Guide to Selection and 
I sc. Hanover. NH: time Share Corporation, 1978. 

Introduction to" computers in education, including instructional -software selection. 
Northwest Regional Educational Laboratory, Equator's Guide for Microcomputer-based Instructional 
lavages. International Council for Computer Education. Eugene, Oregon: 1982. 

The guide uses MicroSIFTs evaluation form, describes the process, and gives detailed 
instructions for using the form. 

Inte "yeXl E U^f i0nS ' InC - C " mpUt " rS in Education: A Practical Guide. Reading, MA: Addison- 
syslemT eV ' !lUa,iCn Crkeria ami strate S ies for implementing computer-based instructional 

ISaaC ve°rsity ' 19«i '" "''^ EdUCOti ' mal M cr(K "^P u ^ ^rams. Fresno, CA: California State Uni- 
A teacher's guide to designing software. Illustrations, sample programs, criteria, etc. 
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Naiman, Adeline. Microcomputers in Education reduction. Chelmsford, MA: Northeast 

Regional Exchange, 1982. 

A beginner's guide to the use of microcomputers in schools. Includes a section on software 

selection, and sample software evaluation forms. 
National Council of Teachers of Mathematics (NCTM). Guidelines fur Evaluating Computerized 

Instructional Materials. Reston, VA: NCTM, 1981. 

Guidelines for evaluation and a form are included. 
Poirot, James. Computers a.^d Education. Manchaca, TX: 1980, 

Includes a discussion on evaluation of software. 

Poirot, James, Kathleen Swigger, and Merridee Heidt, Evaluation Guide for TABS Related 
Courseware. Houston, TX: 1981. 

This book is designed to aid Texas teachers in evaluating software related to state- wide assess- 
ments of basic skills achievement. It includes an evaluation form which allows reviewers to obtain 
a quantitative measure of software effectiveness, 
Pitts, Marcella. The Educator's Unauthorized Microcomputer Survival Manual. Washington, D.C.: 
Council for Educational Development and Research, 1982. 
A beginner's guide which contains questions for reviewing software. 

Strohmenger, Todd. Guidelines for Selecting and Developing Secondary Remediation Software. 
Charleston, WV: Appalachia Educational Laboratory, 1983. 

In disk format, this publication is designed to aid in assessing software for students with develop- 
mental problems other than physical or mental disabilities. 

Texas Education Agency. Guide for Selecting A Computer-Based Instructional System. Austin, TX: 
Texas Education Agency, 1982. 

Contains guidelines for software selection, 
Vann, Eric G. Microcomputers in the Classroom: A Practical Guide for Educators. Glen Ellen, IL.: 

Institute for Educational Research, 198' 
Willis, Terry an'^ William Danley, Jr. Nailing Jelly to a Tree: A Guide to Educational Software. 

Beaverton, OR: Dilithium Press, 1981. 



Directories 

The Addison- Wesley Booh of Apple Computer Software, 19X2 
The Book Company 
16720 Hawthorne Blvd. 
Lawndale, CA 90260 

Describes and evaluates all types of Apple software. 

American Peripherals 
122 Bangor Street 
Lindenhurst, NY 11757 

Two editions; one lists 1200 educational programs for PET; the other lists 400 for VIC-20, 

The Apple Software Directory, Volume Three: Education 
WIDL Video 

524") West Diversey Avenue 
CI. ago, IL 60639 

Describes and indexes by subject, Apple educational software from more than 400 
vendors. 
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Atari I*rogram Exchange 
Atari, Inc. 
P.O. Box 427 

155 Moffett Park Drive, B-l 
Sunnyvale, CA 5)408(5 

Quarterly editions. Atari's compilation of user-written software. Includes special education 
section. 

CIE Software .\ews 

Computer Information Exchange 

Box 159 

San Luis Rev, CA 92068 

A newsletter with a continuously updated directory of software, hooks, and hardware news. 

Classroom Computer News Directory of Educational Computing Resources 
Intentional Educations, Inc. 
341 M" Auburn Street 
Watertown, MA 02172 

Yearly, A reference arranged by category, state, region, type of computer. Lists sources of 
software reviews. 

Commodore Software Encyclopedia 

Commodore Business Machines 

Computer System Division 

Systems Marketing Group 

681 Moore Road 

King of Prussia, PA 19406 

Lists Cor-modore software in seven categories, incluc 1 :?^ -.Ji. ation. 
Curriculum Product Review 
530 University Avenue 
Palo Alto, CA 94301 

Lists texts, AV mater . , h;; and software. 

Educator's Handbook and *>oft i\ise 7 v:rM';.~v 
Vital Information, Inc. 
7899 Maslin Drive 
Overland Park, KS 66204 

Lists evaluated educational programs for the Apple. Includes articles on microcomputer applica- 
tions in education. 1 

Huntington Computing Catalog 
P.O. Box 1297 
Corcoran, CA 93212 

Lists educational and noneducationai p:„Krams. 
IDEAS 

ECS (MKM/M40) 
Digital Equipment Corporation 
200 Forest Street 
Marlboro, MA 01752 

Lists educational software for DEC mainframe computers. 

Index to Computrr-Bas- d Learning, 1981 Edition 

Anastasia Wang, ed. 

Instructional Media Laboratory 

University of Wisconsin 

P.O. Box 413 

Milwaukee, WI 53201 

Lists almost 5000 educational programs. 
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Instant Software 
80 Pine Street 
Peterborough, NH 03458 

For TRS-80, Apple, PET, TI-99/-1 .-.wd Atari 800 microcomputers. 

Instructors 1982-8:1 Computer Director for Schools 
AUn: Elsa Silander 
P.O. Box 6099 
Dulut'h, MN 55806 

Includes articles on software selection and lists of educational software grouped by curricul 
area, machine compatibility, and publisher. 

International Microcomputer Sjftirarv Directory 
Imprint Software 
420 South Howes Street 
'Fort Collins, CO 80521 

Lists microcomputer software in all areas, including education. 

K- 12 Micro Media 
172 Broadway 
Woodcliff Lake, NJ 07675 

March 

280 Linden Avenue 
Brandon, CT 06405 

Lists tested educational programs for Apple, Atari, PET, and TRS-80. 

Micro Co-Op Newsletter 

P.O. Box 432 

West Chicago, IL 60815 

Bimonthly newsletter which ;,rovie:; </ -are listings and d< riptive comparisons 
programs. 

Microcomputers Corporation Catnlog 
\\A Maple Avenue 
P.O. Box 8 
Armonk, NY 1050 

Listing of computer accessories and oftware; many programs ^.re educational. 

Minnesota Educational Computing Consortium ( MECOJnstru :*H>ncJ Computing Catalog 
2520 Bn f riway Drive 
St. Paul, MN 551 13-5199 

A catalog of educational courseware for Apple II and Atari computers. 

Opportunities for Learning, Inc. 
Dept. L-4 

8950 Lurline Avenue 
Chatsworth, CA 9131 1 

Klementary through • oiiege level. 

tJueue 

5 Chapel Hill Drive 
Fairfield, CT 06432 

Catalogs educational software for Apple, Atari, PET, and TRS-80. 

Radio Shack TRS-H0 Educational Software Sourcebook 
From: Radio Shack Stores 

Describes software available for TRS-80 microcomputers in eleven subject areas. 
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o 

Reference Manual for the Instructional Use of Microcomputers 

JEM Research, Discovery Park 

University of Victoria 

P.O. Box 1700, Victoria, BC 

CANADA V8\V 2Y2 

Indexes and evaluates educational programs for the Apple II. 
School Mi crowure Director^' 
Dresden Associates 
P.O. Box 246 
Dresden, ME 041W2 

Lists and describes educational programs for Apple II, Atari, PET, and TRS-80. 
Scholastic Microcomputer Instructional Materia!., 
904 Sylvan Avenue 
Englewood Cliffs, NJ ()76;}2 

All programs have passed educator evaluation tests. 
The Software Directory 
Software Central 
P.O. Box :H)424 
Lincoln, NK 6r<f>0U 

Lists and briefly describes programs for various microcomputers. 
Sources for Courses 
' r \LMIS 

llo North Oak Park Avenue 
Oak Park, ID B();«)I 

Annually lists educational programs for Kindergarten through College. 
Starheh Software Directory 
1 1990 Dorv,ett Road 
St. Louis, \\i i o-UM.'i 

Describes over 1000 Apple-compatible programs. 

Swift'., Directory of Educational Software, Apple II Edition 
Sterling Swift Publishing Co. 
1000 Kortview Road 
Austir., T>; T.S704 

Desv 'ioes commercial and noncommercial educational programs for the Apple. 
Trxa* lustrum? Home Computer Program Library, D)S2 
From: Texas Instruments Dealers 

Lists software for the Texas I r M ts microcomputer. 

[ 'mt'.S 

2o20 Broadway Drive 
S\ Paul, MN 551 \ :\ 

Lists Apple and Atari Software developed by educators for the Minnesota Educational 
( omputing Consortium (MECC). 

VanLoees Apple Will Software Directory, Vol. // 
Vital Information, Inc. 
7899 Mastin Drive 
Overland Park, KS 66204 

A comprehensive directory of Apple software which includes an educational software 
section. 
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Articles 

Anderson, Ronald, "Courseware Deserves Evaluation Including Peer Review," AEDS Monitor, 20 
(April-June 1982): 10-12. 

Barry, Tim. "Considering Software Reviews," InfoWorld, 3 (Nov. 2, 1981):20-22. 

Ban>, Tim, "Considering Software Reviews II," InfoWorld, ;\ (Nov. 9, 1981):20-21. 

Bingham, Margaret, "Software Evaluation: A Coordinated Effort Needed " AEI)S Monitor, 20 (April- 
June 1982): 10- 12. 

Caldwell, Robert M., "Guidelines for Developing Basic Skills: Instructional Materials for Use with 
Microcomputer Technology," Educational Technology, 20 (October 198()):7-12. 

Cohen. Vicki Blum, "Criteria for the Evaluation of Microcomputer Software," Educational 
Technology (January 1983): 9-14. 

Dyer, Susan R. and Richard C. Forcier, "How to Pick Computer Software," Instructional Innovator, 
27 (September i982):38-40. 

"ERIC Reports," Educational Technology, 22 ( A* - "il i l J82):(51-62. 

Forman. Denyse, "Courseware Evaluation for Curriculum Integration," AEDS Monitor. 20 (April- 
June 1982). 

G. .i, Doug, "Green's Eleven Commandments for Educational Software," InfoWorld, 3 (November 
23, 19811:5 

Grimes. Lynn. "Computers are for Kids: Designing Software Programs to Avoid Problems of 
Learning" Tewhing Exceptional Children, (November, 1981):49-53. 

Hakansson, •> c e, "How to Evaluate Educational Courseware," The Journal of Courseware Review, 1 
(1982). 

Hannaford, Alonzo E. and Florence M. Taber, "Microcomputer Software for the Handicapped: 
Development and Evaluation," Exceptional Children, 49 (October 1982): 137-144. 

Hilgenfeld, Robert, "'Checking Out' Software." The Computing Teacher (November 1981). 

Holznagel, Donald, "Courseware and Software Needs in Education," The Computing Teacher, 8 (May 

1981) :36-:*8. 

Holznagel, Doix.id. "What Courseware is Right for You?* 1 'Microcomputing Magazine (October 

" 1). 

Hunter, Beverly, "How to be a Program Critic," Eleetranic Learning, / (January-February 

1982) :G r ). 

Judd, Ccrothy, and Robert Judd, "Evaluation of Instruct -na! Programs for Microcomputers," Edu- 
cational Computer,* 2 (March-April 1982): 16- 17. 

Kansky, Bob, »-\!iiam Heck, and Jerry Johnson, "Getting Hard-Nosed about Software: Guidelines for 
Evaluating Computerized Instructional Materials," The Mathematics Teacher, 74 (November 
1981). 

Koetke, Walter, "Software Shopping," Kilobaud Microcomputing (July 1981). 

Lathrop, Ann, "Building the Software Collection," Educational Computer, 1 (November-December 
19S1):23-K 

Lathrop, Ann, "Microcomputer Software for Instructional Use — Where a ritical Reviews?' 

The Computing Teacher, / (September-October 1981): 14- 15. 

Lathrop, Ann, "Software . . . Previewing and Reviewing," Educational Computer, J (Sep 
October 1981). 

"Level-Headed Guide to Software Evaluation," Classroom Computer News, 1 (July-Augus> 
22-23. 

Moursund, David, "Some Thoughts on Reviewing Software/ 1 The 'Computing Teacher, 7 (June**, 
1980). 
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N ' l,<,t Iqsn"? d ' a,Ul Ueborah VVise ' " Ht ' 1 ' ,i "K Teachers Pick Software," InfoWorki ,7 (Dec. 7, 

Peters. I lamld -I., and Molly H. Hepler. " Reflect ions on Ten Years of Experience." A EDS Monitor X, 
(April-June 11)82): ID- 12. 

•'Quality Software: How to Know When You've Found It!" Electronic Learning 1 (November- 
Deeeinoer UW l):.{3-.{(i. 

Kobiyer, M.D.. 'Guidelines for Evaluating Computerized Instructional Materials/' £</(«•,,*/,,,,,,/ 7Vr/i- 
nnhxy. 22 (February 1982):44-4". 

Kobiyer. M.D.. "When is it -Good Courseware?': Problems i„ Developing Standards for 
Microcomputer Courseware," Educational Technology, 21 i October 198l):47-fvl 

Wade T.E.. .Jr.. 'Evaluating Computer Instructional Program.- .• d Other Teaching Units" 

hducatiin: : I fchnolnny, 20 ( \ov<-nibe.". 1980). 

Wager, Walter, 'Issues in the Evaluation of pactional Computing Programs," Educational Com- 
puter, 1 (September-October 1981 ):2r 

Wager, Walter, "The Software Evaluation Dilemma/' AEUS Monitor, 20 (April-June 1982):5-6. 

Watt. Molly, "MnUm: r as -. r nr Software Evaluation," Computing Teacher, 9 (May 1982): 20-22. 

Wist- Deborah, "Softwai.. is the Key to Educational Computing, Teachers Say/' Info World :i 
(November 1 981:17 + . ' 



Clearinghouses and Information Centers 

Apple Computer Clearinghouse for the ! fandlcapped 
Prentke Pomich Comp any 

R.n. 2, r.o. Box 19 i 

Sl.rjve .OH 44(>7(i 

A source of information about Apple software being developed for handicapped people. 
Apple for the Teacher 
c/o Fed Perry 
5848 Kiddio Street 
Citrus Heights, CA ^f>bl() 

Operates the National Computer-Assisted Library for the Apple, a Software collection of inter- 
national scope. Publishes a newsletter containing reviews. 

Boston Computer Society (BCS) 

Educational Resource Exchange 

Three Center Plaza 

Boston, MA 02108 

Maintains an educational resource exchange which disseminates information on software selec- 
tion. 

California Library Media Consortium 
San Mateo County Off;,v of Education 
'•VM\ Main Street 
Redwood City, CA 9-: : 'C:; 

Organized in 198 1 by f,.| library media .specialists. Publishes COURSEWARE REVIEWS All 
reviews arv written by educators and based on classroom use of software. 
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California Software Clearinghouse 
Office of Staff Development 
State Department of Education 
7i> I Capitol Mall 
Room 634 

Sa cram e n t o, C A H . r ) S 1 1 

The Clearinghouse supports fifteen teacher Education and Complin.. -\*ry (Tl^CC) around 
the state, provides staff development, print resources, a collection of si-.. • ."..re and disseminates 
evalutions of software. 

CONDUIT 

P.O. Box 388 

Iowa City. I A .V22-U 

Tests, reviews and distribiiu c. . -ft vare; the focus is on software foi higher education. 

Computer Technology Task Force 1 
Superintendent of Public Instruction 
#7510 Armstrong St.. SAY. #FG 11 
Tumwater, \VA 98504 

This task force operates a telephone exchange for Washington educators which can he reach. . at 
206/753-2858 or 206/753-67^7. The task force has also published six handbooks on educational 
technology. One handbook is devoted to software evaluation. 

Computer- Using Educators (CUE) 
17 76 Educational Park Drive 
San Jose, CA 95133 

Operates a software li^r-^y, a microcomputer demonstration center, and an in-service training 
program. 

Educational Products Information Exchange Institute (EPIE) 

P.O. Box 620 

Stony Brook, NY 1 1790 

An educational consumer advocacy gnr v/hich, in conjunction with the Microcomputer 
Resource Center at Columbia University 'i v:ors College, publishes detailed critical reviews of 
commercially available educational software. 

Florida Center for Instructional Computing 
College of Education 
University of South Florida 
Tampa. FL 33620 

Funded in part through the Florida State Department of Education, the FCIC serves Florida 
educators by compiling software reviews and maintaining an educational software index. 

Helping Schools and Community Colleges to Choose Microcomputer Courseware 

c/o Dr. Vicki Blum Cohen 

Microcomputer Resource Center 

Box IS, Teachers College 

Columbia University 

New York, NY 10027 

This project is producing detailed reviews of courseware in math, science, and communication 
skills designed for elementary through community college level;;. KPiK ^nbl^hes the reviews 
periodically. 

Instructional Materials Division 
Department of Education 
State of New Mexico 
Santa Fe, NM 87501 -'21 86 

The Instructional , ":j*' l rials Division reviews educational software and pubhVies ■ l : s! of state 
adopted softwan . 
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Materials Review and Evaluation Center 
North Carolina Dept. of Public In miction 
Raleigh, N'C 2 7 (> 1 1 

Reviews educational software ;U ul publishes a list of highly rated pieces, "Advisory List of 
Instructional Media." which is sent to all 50 state departments of education in the U,S, and to 
schools throughout North Carolina. 

Michigan Association for Computer Users in Learning (MACCL) 
:Wf>00 Van Horn Road 
Wayne. Ml 48184 

Collects and reviews public domain educational software for the Apple. Pub' --lies revi ,,f 
educational programming. 

Micro Co-op 

P.O. Box -l.rj 

West Chicago, IL (5018") 

A software cooperative that distributes Apple and Atari software to members at reduced rates 
Its bimonthly newsletter describes and eompares programs. 

Microcomputer Education Applications Network (MEAN*, 
2o(i North Washington Street 
Falls Church, VA 22041) 

Aids educators in developing and selling software. Publishes a quarterly newsletter which 
contains information on software services. 

Microcomputer Resource Center- 
Teachers College. Columbia University 
f>2f> W. 121st Street 
New York, NY 10027 

Maintains a collection of hardware, software, journals, and books, which educators are welcome 
to use. Runs seminars and workshops on microcomputer applications in the schools. 

Microcomputer Software and Information for Teachers ^MicroSIFT) 

Northwest Regional Educational Laboratory 

•MH) SW fith Avenue 

Portland. OR 97204 

Disseminates descriptive and evaluative information about educational software. Its reviews are 
available through state and local education agencies, various periodicals, and the RICE database 
the Bibliographic Retreival Services Computerized information system. 

N.sNonal Council of Teachers of Mathematics (NCTM) 

1HM) Association Drive 

Huston, VA 20U91 

Publishes software reviews in its various periodicals. Developed GuUMbws for Evaluatim Com- 
puterized Instructional \!(it,» \ 

SOFTS WAP 
c/o Ann Lathrop 

San Mateo Co.i n ; v Office „ r iv^.eatio:, 
XVA Main Street 
Redwood City. CA 94()(;:; 

Collects, evaluates, and modifies educational programs, and makes them available free of chaw 
to educators who copy them at the center. Operates a software exchange which allows any 
educitor who contributes a program to request one in exchange. Sells public domain software at 
low cost. 
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Technical Education Ke-earch ('(Miters, Inc. (TKRO) 
S Kliot Street 
Cambridge, MA O'Jl.'W 

Conducts workshops and provides information services relating to educational computing. A 
software review service is in the planning stage. 

Periodicals And Reports Devoted To Software Reviews 

Computer Software; A Manual for Teachers 
Teacher Center for Montana 
2 1 T> S. 6th Street West 
Missoula, MT M>801 

A collection of reviews of software in Language Arts, Science, Soc ; 1 ' :. :> h' 

Courseware Report Card 
150 West Carob Street 
Compton, CA 902 'JO 

f> issues per year. Separate elementary and secondary editions review education. ■ s f ■• 

Apple, Radio Shack, Atari, Commodore, and Texas Instruments. 

Digest of Software Reviews: Education 
L'Ml Bulldog Lane, Suite C 
Fresno. CA 93710 

Four issues per year; describes 50 programs per issue. 

Dvorak's Softwar" Review 
704 Solano Avenue 
Albany, CA 94706 

Plight issues per year. 

■Journal of Courseware Review 
TLe Apple Education Foundation 
'i' 1 "e.\. • M . ''ani Avenue 
r.joe/t; .o. CA 95014 

o ssues per year. Contains signed critical reviews of coram* rcial educational programs for the 
Ap:>le II. 

MAC IT. -Journal 
c/o Larrv Smith 
Wavne County JSP 
P.O. Mov SOY 
Wayne, MI 4 8184 

Occasional. The Winter 1981 issue was devoted to software reviews. Focuses on Apple II soft- 
ware. 

Microcomputer Courseware/ Microprocessor Games. 

EPIE Materials Report 9S/V9m. 

from; EPIE Institute 

P.O. Box 620 

Stony Biook. NY 11790 

Includes reviews oi commercial educational software packages. 

Microcomputers in Education 
Queue, Inc. 

a Chape i Hill Drive ; 
Fairfield, CT 064-T2 

Monthly. Describes new educational programs, reviews software, and summarizes software 
reviews from other sources. 
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MICRO-SCOW. 
•I KM Research 
Discovery- Park 
University of Victoria 
P.O. Box 1700 

Victoria, BO VHW 2Y2 CANADA 

Monthly. 

Peeling I! 

P.O. Box 188 

Las (Alices, NM 88004 

Nino issues year year. Describes and ew,\. lCS commercially available software for the 
Apple II. 

I Hp rlint' 

P.O. Box ,'188 

Iowa City, IA 52244 

Twice yearly. Describes software reviewed and tested by the Conduit Clearinghouse. 
/ >l I US EH % S MA OA ZINE 
P.O. Box 466 
Kl Dorado >; CA 9562:1 

Quarterly. Reviews software for TRo-80, Apple, and Atari. 

School Micro ware Reviews ' 
Dresden Associates 
P.O. Box 246 
Dresden, ME 04,'U2 

Semi-annually. Includes detailed teacher evaluations of educational programs for the Apple 
Atari, PET, and TRS-80. 

So [tic arc Review 
Meckler Publishing 
f>2() Riverside Avenue 
Westport, CT 06880 

Quarterly. Reviews software designed for school and library use. 

7 % AL MIS Courseware Ra t intf s 

II o North Oak Park Avenue 
Oak Park, IL fSOWH 

Contains teacher produced evaluations and ratings for commercially published educational soft- 
ware. 



Periodicals Containing Software Reviews 

AEDS Monitor 

1201 16th Street, WW. 

Washington, DC 2(HK*6 

Four issues per year, with an average of two reviews per issue. 
Applesauce 
P.O. Box 598 
Venice, CA 90291 

Six issut-:; per year. Contains reviews of Apple software. 
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Arithmetic Teacher 

National Council of Teachers of Mathematics 
1906 Association Drive 
Reston, VA 22091 

Nine issues yearly. 

BYTE 

70 Main Street 
Peterborough, NH 03458 

Monthly. Includes detailed discussions of new software. 

Call — A. P. RLE. 

Apple Puget Sound Program 

Library Exchange 

304 Main Avenue S., Suite 300 

Renton, WA 98055 

Monthly. Contains Apple software reviews. 

Classroom Computer News 
Intentional Educations, Inc. 
341 Mt. Auburn Street 
Watertown, MA 02172 

Six issues yearly. Includes a regular review section focusing on educational software. 
Compute! 

515 Abbott Drive 
Broomall, PA 19008 

Monthly. Frequently features educational programs. 

77i£» Computing Teacher 

Department of Computer and Information Science 
University of Oregon 
Eugene, OR 97403 

Nine issues per year. Regularly includes reviews of educational software. 

Creative Computing 
P.O. Box 789-M 
Morristown, NJ 07960 

Monthly. Occasional special educational issues focus on software for school use. 

C\UE. Newsletter 
P.O. Box 18457 
San Jose, CA 95158 

Six issues per year. Regularly reviews educational software. 

Educational Computer Magazine 
P.O. Box 535 
Cupertino, CA 95015 

Bimonthly. Includes software reviews in every issue. 

Educational Technology 
140 Sylvan Avenue 
Englewood Cliffs, NJ 07632 

Monthly. Regularly reviews commercially available educational software. 

The Electronic Classroom 
150 West Carob Street 
Compton, CA 90220 

Monthly. Contains detailed evaluations of software designed for classroom use. 
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Electronic Education 

Electronic Communications, Inc. 

Suite 220 

1311 Executive Center Drive 
Tallahassee, FL 32301 

pa e c n kige UeS yeady ' EaCH iSSUe C ° n ' ainS 3 d6tailed reVi6W ° t a C ° mputer s y* tem or instructional 

Electronic Learning 

Scholastic Inc. 

902 Sylvan Avenue 

P.O. Box 2001 

Englewood Cliffs, NJ 07632 

mirocompul a r C s h C ° ntaiRS ^ of educati °-' applications of 

Hardcore Computing 
14404 East D Street 
Tacoma, WA 98445 

Quarterly. Contains program listings, reviews and discussions of issues relating to software 
Info World 

375 Cochituate Road 
P.O. Box 880 
Framingham, MA 01701 

Weekly. Geared toward the beginner. Contains detailed product reviews. 
Interface Age 
16704 Marquardt Avenue 
Cerritos, CA 90701 

Monthly. 

Journal of Computers in Mathematics and Science Teaching 
Association for Computers in Mathematics and Science Machine 
P.O. Box 4455 * 
Austin, TX 78765 

Action. C ° ntainS SOftW3re HStingS ' r6VieWS ' discussions °f computer use in 

Journal of Learning Disabilities 
1331 E. Thunderhead Drive 
Tucson, AZ 85718 

Quarterly. Publishes reviews of software designed for handicapped populations. 
Kilobaud Microcomputing 
80 Pine Street 
Peterborough, NH 03458 

Monthly. 
Mathematics Teacher 

National Council of Teachers of Mathematics 
1906 Association Drive 
Reston, VA 22091 

™t e ^ e7'^T^ E H mpha T eS PraCtiCal 3idS f ° r mathematics educ ation in secondary and junior 
college level. Includes software reviews. 
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80 Microcomputing 
80 Pine Street 
Peterborough, NH 03458 

Monthly. Concerned with TRS 80 information only. 

Microcomputer Digest 
103 Bridge Avenue 
Bay Head, NJ 08742 

Monthly (except August). Information and comparisons of hardware and software for teachers 
and educational administrators. 

Nibble 

P.O. Box 325 
Lincoln, MA 01773 

Eight issues/year. Program listings, tips, and software reviews. 

Personal Computing 
P.O. Box 1408 
Riverton, NJ 08077 

Monthly. 

Popular Computing 
70 Main Street 
Peterborough, NH 03458 
Monthly. 

T.KE. Journal 
P.O. Box 992 
Acton, MA 01720 

Bimonthly. Focuses on educational application of technology. Contains reviews of software, pro- 
jects and publications. 

Window 

469 Pleasant Street 
Watertown, MA 02172 

A magazine on disk, issued 5 times per year. Allows users to try out software. 
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Computer Accessible Databases 



I h fL BRS a f d Di f- 0g 0nHne biblio * ra P hic search services provide access to databases containing 
soitware information. & 

Databases Accessible through BRS 

SPIF (School Practices Information File) 

mI^Spt ° re f. than 1,5 °,° deScri P tions of educational software. Includes the MARCK and 
MicroblFT software catalogs. 

RICE (Resources in Computer Education) 

Includes descriptions of educational software and information concerning software producers. 
Bibliographic Retrieval Services 
1200 Route 7 
Latham, NY 12100 

Databases Accessible through Dialog 
International Software Databases 

Contains descriptions of all types of software, contributed by vendors throughout the world 
Includes references to independent reviews. 
Microcomputer Index 

Indexes more than 25 English language microcomputer periodicals. 

Dialog Information Retrieval Services 

3460 Hillview Avenue 

Palo Alto, CA 94304 
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RESOURCE MATERIALS FROM THE EXCHANGE 




This resource book resulted from the cooperative work cf key 
ed'jcat^on professionals in tha seven state region of the Northeast, the 
staff and financial support of the Exchange and the contractual services 
of Technical Education Research Centers in Cambridge, Massachusetts. As 
a service to the region NEREX has disseminated at no charge over 3000 
copies of this resource. The seven state departments of education in our 
region helped distribute the book to teachers and other educators who are 
promoting the effective use of technology in education. As a personal 
resource and as an in-service text book Microcomputers in Education: An 
Introducti on is helping educators plan ways to successfully use computers 
as instructional aids. Since the demand for this book h&s been so high 
we have printed additional copies for sale. You can order additional 
copies with the form on the next page. 



This resource book was produced by the New York State Education 
Department's Center r or Learning Technologies under a State Priority 
Grant from the Northeast Regional Exchange, Inc. New York has 
disseminated over 10,000 copies of this book within their own state and 
NEREX has disseminated over 1,000 copies to other states in the region. 

Computer Literacy: An Introduction , provides a national, state, and 
local perspective on computer 1 iteracy. Featured are sections describing 
results of a national survey of computer experts, state and loca'< 
activities, reprints of key journal articles depicting varying views, 
materials about literacy for school personnel and the public, materials 
about computer literacy for students and annotated bibliographies. 

Since the demand for this resource book has been so high we have 
printed additional copies for sale. You can order additional copies with 
the form on the next page. 



Technology Programs That Work is an National Oiffusion Network 
catalog of exemplary 6'ducational programs validated on the basis of their 
effective use of technology. This catalog was developed and printed at 
the private expense of The Northeast Regional Exchange. Inc., a 
not-for-profit educational service organization. 

The catalog contains descriptions of nineteen National Oiffusion 
Network Technology Programs including the four new NDN Technology 
Lighthouse Projects. Please use the form on the next page to order 
additional copies. 
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northeast regional exchange 

101 MILL ROAD, CHELMSFORD, MA 01824 • 617/256-3987 



Microcomputers In Education: An Introduction 





Quantity 


Amount 


1 Copy § $6.00 






10 or More Copies § $4.50 






100 or More Copies § $4.00 







Computer Literac y; An Intcoductio p 





Quantity 


Amount 


1 Copy § $6.00 






10 or More Copies 9 $5.00 






100 or More Copies § $4.50 




1 









Technology Programs That Work 





Quantity 


Amount 


1 Copy # $3.50 






10 or Mere Copies § $3.00 






100 or More Copies § $2.50 







TOTAL AMOUNT 



All prices include shipping costs. 

NER E x' d inc S . leSS ^ $25 - 00 » ust »>• P^paid. Make Cec.cs payable to 



Name: 

Organization: 
Address : 

Send to: 



NORTHEAST REGIONAL EXCHANGE, INC 
101 MILL ROAD 
CHELMSFORD, MA 01824 
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RAD Speaks : Ml crocomputers and Educati on is the proceedings from an 
invitational conference of state education agency and local school 
district personnel held at SEDL in October, 1981. A service of the 
Regional Exchange Project, the conference was designed to provide 
participants with a forum for discussing major issues such as the role 
of stattj 'Education agencies, teacher training in microcomputer use, 
and evaluation of software. The book includes a discussion of major 
issues to consider in microcomputer use; computing competencies for 
teachers; ERIE and MicroSIFT approaches to software evaluation and 
information on approaches to microcomputer use in six school districts 
and four state education agencies (Arkansas, Louisiana, Oklahoma, and 
Texas). $6.00 



Planning for "Leadershi p . i n Microcomputers" A Networking Project 1s 
the proceedings from a conference of state education agency personnel 
held p.c SEDL in January, 1982. A service of the Regional Planning and 
Service Project, the conference was designed to assist SEAs 1n moving 
qui ckly into a leadership rol e in compute r1 zed instructional systems 
by forming a network of state agencies computer companies, and SEDL. 
This book contains discussions of the following topics: What Computer 
Related Competencies are Needed by SEA Staff Members? How Can Projet* 
Activities be Most Effectively Managed in Each State? What are the 
Optimum Positions and Actions for SEAs to Take to Achieve Leadership 
1n Microcomputer Use for Education? plus summaries of key T 
presentations by Norman Bell, J1m Dugan, and J1m Poirot. $5.00 




Leadership In Microcomputers Meeting II: Models for Planning and Use 
7s Ene report of an invitational meeting or state education aoenc y 
personnel held at SEDL 1n September, 1982. A service of the Regional 
Planning and Service Project, the meeting was a follow up to the 
January 1982 "Leadership 1n Microcomputers" meeting described above. 
This book contains reports from six SEAs (Arkansas, Louisiana, 
Mississippi , New Mexico, Oklahoma', and Texas) ; discussions of state 
and local educational u:es of technology (Including MECC, the 
California SEA, the Lexlr .on MA schools, and -he Lyons Township, IL 
schools); and appendices. $5.00 



The Educator's Unauthorized Microcomputer Survival Manual was 
produced in 1982 by the Council For Educational Development and 
Research, a non-profit association of educational research 
organizations to which SEDL belongs. The manual is designed for 
anyone fac 1 ng dec i s i ons about whe re , how , and whet her to use 
microcomputers for instruction, The manual's view of technology 1s 
-different from what you'll hear from technology enthusiasts 1n 
government and from computer manufacturers backing legislation to 
Increase the number of microcomputers in the schools." The book 
contains such discussions as "survival tactics," computer jargon, 
deciding how to use microcomputers, and selecting software. $2.00 
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